This message was deleted.

I use env0 and it can enforce approval policies... so if certain resources are changed you can't approve a plan. I've also used resource locks in Azure. I code the locks in TF. I'll run a plan, see the resources that it affects, delete all the necessary locks, in the UI, then run the apply. After which I run the environment again and rebuild the locks.

Awesome, thanks. We use spacelift which looks similar to env0, but it doesn't stop people from applying a plan directly to the environment if they have the appropriate permission. The resource locks strategy is interesting, I wonder if GCP has something similar.

I've never used GCP, is there a privilege escalation pattern you can use to lock reading cloud environment state files? Even if it's always approved, it could work like a fire break if you run the wrong environment, then get a permission read error.

I've been reading about some people using groups to manage priv esc, so normal users are unprivileged, but if they need to gain privileged access for clickops, use a tool to temporarily add the user to a privileged group.

Sharing my 2 cents on this. You are trying to avoid accidental TF Apply by users. IMO, the only way you can guarantee avoiding such a situation is when users only have READ permissions. Your CI/CD systems should be the only ones with access to perform Cr/Upd/Del. You can implement OIDC between your CI and Cloud Providers. If you have a monolithic repo model, you will give Complete Admin control to the CI/CD associated with that repo. If you microservice style repositories maintaining different resources - you can control which repo gets what permissions (more favorable - but repos are hard to manage and references will look scattered)

For sure. Working through locking the environments down as well, but it takes a bit of time. In the meantime, looking to put some more barriers in place.

Like @GP , using a CI/CD with a correct review / approval check can also help. That's what we do in our side for each terraform execution. Thanks to this process we perform a syntax check and a plan to be sure changes are correct and then only people with merge rights can merge the code to do "terraform apply". TF states files ate stored on S3. We use terragrunt during CI/CD.

we’ve used runatlantis.io for the last 4 years at kubefirst and it’s been great

We are using github actions to get one more level of approval for the relevant environment.