https://platformengineering.org logo
#terraform
Title
# terraform
l

Lights On

08/24/2022, 3:58 PM
#terraform Has anyone successfully setup composer 2 using terraform on GCP?
j

John Baird

08/24/2022, 6:06 PM
I literally just did this like a week ago? How can i help? 😛
p

Prune

08/25/2022, 7:56 PM
we’re also doing that in my company - though I did not build the thing, I can have a look if you have precise questions…
l

Lights On

08/25/2022, 7:59 PM
Awesome @John Baird and @Prune. Can you provide the template, that your team use to put this in place.
j

John Baird

08/25/2022, 8:02 PM
Copy code
resource "google_composer_environment" "airflow" {
  name    = var.project_id
  project = var.project_id
  region  = var.region
  config {
    private_environment_config {
      enable_private_endpoint          = true
      enable_privately_used_public_ips = false
      # ip_allocation_policy {
      #   cluster_ipv4_cidr_block  = var.cluster_ipv4_cidr_block
      #   services_ipv4_cidr_block = var.services_ipv4_cidr_block
      # }
    }

    software_config {
      env_variables = var.env_variables
      image_version = "composer-2-airflow-2"
    }
    workloads_config {
      scheduler {
        cpu        = 0.5
        memory_gb  = 1.875
        storage_gb = 1
        count      = 1
      }
      web_server {
        cpu        = 0.5
        memory_gb  = 1.875
        storage_gb = 1
      }
      worker {
        cpu        = 0.5
        memory_gb  = 1.875
        storage_gb = 1
        min_count  = 1
        max_count  = 3
      }
    }
    environment_size = "ENVIRONMENT_SIZE_SMALL"
    node_config {
      network         = google_compute_network.airflow.id
      subnetwork      = google_compute_subnetwork.airflow.id
      service_account = google_service_account.airflow.name
    }
  }
}
p

Prune

08/25/2022, 8:02 PM
I’ll come back to you shortly, in a call
but there’s a “public” module for it
l

Lights On

08/25/2022, 8:02 PM
Thanks @John Baird
we wrapped that under our own module (we’re using Terragrunt)
but this should be enough
l

Lights On

08/26/2022, 12:25 PM
Thank you @Prune
30 Views