Ho do you test infrastructure changes (e.g running terraform apply) ? Are you running them in your PRs? Do you have different AWS/Cloud accounts? How do you protect that 2 PR that are touching terraform files become inconsistent, because you share the Terraform status?

I've written an article about this topic some time ago: https://dev.to/mraszplewicz/testing-in-infrastructure-as-code-and-why-terraform-may-not-be-the-best-option-3k2i My short answer is: I use Pulumi or AWS CDK and write unit tests. But also I know that some integration tests (with the real cloud) are required so in the integration tests I am using a randomly generated UUID everywhere and I also have good cleanup code in the

finally

block (or

defer

in Golang). And all the integration tests are being executed on a sandbox AWS account. Also, I am executing aws-nuke (https://github.com/rebuy-de/aws-nuke) on this account from time to time. I am not running integration tests in PRs (yet?). However, adding the UUID should do the trick for PRs too.

How do you protect that 2 PR that are touching terraform files become inconsistent, because you share the Terraform status?

The workflow I’ve seen at several places is something like: • PR does a plan/preview step against the real infrastructure • only

main

branch deployments actually get deployed to production • Any automated tests we write against the infra are used to build out the terraform/pulumi in a sandbox account where we run it manually. • Those tests do get run against production after the deployment, but they don’t trigger rollbacks if they fail (just because IAC rollbacks can get very hairy very easily) but do fail the pipeline and alert the teams they have to fix it

GH PR triggers a TF plan and run tests like OPA and other policies. Merge into main triggers an apply to the infra. Since only merges into main run an apply there should be no conflicts between branches writing to the same state file