Hello Team, We have been following the Gitops model with Terragrunt/terraform and bunch of other tools which is a declarative way of provisioning resources in Cloud. The Engineer puts the desired config in a yaml , raises a PR and once the PR gets merged a pipeline is triggered that creates the resource. We are looking forward to a more imperative way which is developer driven. For eg, the developer provides the bare minimum inputs required for creating any AWS service (could be a cli interface or a UI) and the infrastructure gets provisioned. 1. What should be the choice of tools for infrastructure provisioning for such a use case?Can Terraform still be used? 2. Should we still follow the Gitiops model and PR approval process? a. Writing a layer to take inputs from the user, raising a PR etc would be cumbersome and also could result in merge conflicts etc. 3. Should desired config still be persisted in Git or should we look at an alternate solution for storing the configs? 4. Should we rather look to use sdk and create API’s? Appreciate any pointers, Thanks.

Hi Sandeep, I can't properly answer all the questions right now but a quick pointer is to check Score: https://www.cncf.io/projects/score/

Hi @sandeep nair - We are building an IDP that can meet your use-case, developer goes onto a Self-Service UI to request new infra (or changes to existing infra) and the IDP makes commits in the background to Git

Check out release.com it might already do all you’re looking for @sandeep nair

@sandeep nair I am curious what you mean forward to a more imperative way driven by developer? IIUC your old way already be developer driven.

it wouldn't make sense to have imperative actions with terraform/terragrunt, that was confusing.

Gruntwork CEO here. We’re the creators of Terragrunt. There are a few thoughts to share: 1. Since you’re already using Terragrunt, you might want to leverage terragrunt catalog, which allows any user to browse a git repo you designate as a Terraform module catalog, and then scaffold a file (based on a template you can customize!) so that the dev only needs to fill in the variable inputs. That’s one way to solve the “create the code” part of what you’re asking. 2. You’ve got a choice of paradigms to consider for the developer self-service experience: a. Some shops use a paradigm where Terraform/OpenTofu/Terragrunt is used to deploy 100% of infrastructure. If that works for you and your team, great! b. Other teams use Terragrunt for “stable” infrastructure like AWS account baselines and VPC configurations, and use a “pull-based” CI system to deploy infra that’s closer to the developers. For example, if you’re running apps in K8s, you can use ArgoCD to automatically “sync” updates to Helm charts. This gives you a GitOps-style approach, but without the “heaviness” of a full-fledged PR review process. I’m curious if any of those suggestions meet the UX requirements you have in mind here?

Thx all for the feedback ..was out for a couple of days ..Let me go through them and get back..🙏

Yeh - We will push to a branch and optionally open a PR: https://docs.flanksource.com/playbooks/actions/gitops/

Yep, that's just how we do inside our company, which we call it intend driven approach.

@sandeep nair:

We are looking to provide an interface to the developers where they can request for infrastructure resources.

To state my bias explicitly, Gruntwork makes Terragrunt, and we have a DevOps platform ourselves. But as you’re seeing there are many ways to solve the dev self-service challenge. To comment on what I’m familiar with, if you want to expose self-service capabilities to developers, then you need to define a catalog of some kind that contains all your company’s blueprints. Then you need to create an interface to that catalog, which can be either CLI, Web, or API. Since you’re already using Terragrunt, using the terragrunt catalog command is a good option for you since it’s free, included, and exposes the CLI interface. Later on, Gruntwork will offer a paid Web UI for that, but we don’t have that today (whereas it sounds like others do). My recommendation would be to try out something simple and quick with a small number of developers and see what kind of feedback you get from them. If you’d like to correspond further with me, you can email me at josh@gruntwork.io. Hope this helps!

Thx @Josh Padnick for the response, We are looking at creating our own UI and enable self service for developer community. Couple of Questions on the catalog - 1)How do you execute the terraform code or repository registered in the Catalog? 2)Do you use sort of tool like Jenkins? 3)Where do you persist the desired config that you typically have in the Terragrung hcl file while using the Catalog?Do you use Git and raise PR’s?

@sandeep nair Would you mind emailing me at josh@gruntwork.io? This way I can cc’ my colleagues to assist with your questions. But the short answer is that you the job of

terragrunt catalog

is to generate your code. You can then use any number of Pipeline solutions to deploy it. Our commercial solution is Gruntwork Pipelines, but there are also open source tools like Atlantis, or many companies build their own automated deployment system on their internal CI system (such as Jenkins). Re: “persisting the desired config,”

terragrunt catalog

uses a boilerplate template to generate a

terragrunt.hcl

file and customizes the

inputs

and

source

properties based on the Terraform/OpenTofu module you selected! The basic flow is: 1. User runs

terragrunt catalog

2. User sees a CLI interface of your catalog, based on your git repo configuration 3. User selects their preferred module 4. Terragrunt generates a

terragrunt.hcl

file based on your boilerplate template and the selected module 5. User can then deploy that change however you decide (Gruntwork Pipelines, Jenkins, or anything else)

hey @sandeep nair, love to hear you are thinking about the persistence behind the UI. I work on the OSS tool Kratix.io that is often used as the API behind Backstage and other UIs as it is a gitops focused workflow engine. So basically the UI just creates a git commit which acts as a request to the Kratix API. Then people run their actions (including terraform) in the Promise Workflows, though if they have big investment in existing tools like enterprise or atlantis then they will trigger those jobs. Hope that helps and good luck with your platform building!