This message was deleted.

I think that it does have some upsides. If you use a centralized place of managing everything, the burden is the context of all of the pieces. Do you want new engineers (onboarding to the org, or building new services) to front load all the context needed to place resources that impact their application, or would you rather free up their time to allow an abstraction over IaC to provide a workflow that is free of context loading.

My thoughts exactly, I wouldn’t want to not allow developers to use IaC to get to something not supported but an abstraction provides the easier path and allows engineers to concentrate on building new services.

Some % will always need/want to go below the abstraction. At Spotify ~25% wanted to read/edit K8s

should be avoided and developers should learn to use the tools

It can be a viable approach, depending on the size and the stage of the company. You will still want some level of abstraction. Otherwise, you will end up with code duplication and maintenance problems as everybody will write their own low level resource definitions. Of course, different tools already provide different levels of abstraction. What particular tools was the discussion about?

The discussion started around some of our terraform, an engineer had added a small YAML file that has a project configuration and when committed to source control ran terraform to setup projects, permissions etc.. in GCP. This was questioned as an abstraction that denied the developer knowledge of what was happening. Counter point was they could still look at the terraform but they didn’t need to learn or fully understand it to get started on their project.

I see, thanks for clarification. In itself, this direction has proven to be valuable for many companies, of course, else we wouldn’t have this Slack channel. Now you seem to have a starting point and if it’s more or less ready to be used, give it a try with a few other developers/teams and gather their feedback. Of course, this requires a bit more time and focus than a hypothetical discussion, but also gives a clearer answer.

Thanks Mike, it’s been used by a few developers and appears liked (more actual feedback is needed of course). This debate was started by a new manger who believes in no abstractions at all over IaC but without any feedback, tangible evidence, experience, etc..

Platforms won’t be built at once, but infrastructure is needed here and now. An excellent middle step is to create easy-to-use Terraform modules with examples and links to real-life implementation. I just wanted to let you know that it works for us. Developers can build IaC with our review and consultancy, and we can build a platform.

We are in an interesting place, the infrastructure and IaC has been built in isolation over the past few years and sits in a mix of Bicep and Terraform modules. A lot of the developers are seeing this for the first time and it’s all new and now the journey of developer experience begins

It was the same on our site. But after six months, it’s clear that the decision paid off. Additional profits: • developers are aware of infrastructure on which they are building solution (no hardcoded resources names)

It REALLY depends on the culture and hiring of the organisation. So it’s not a once size fits all, I’ve been at places where all engineers were at a standard where I would have been confident in handing direct access to IAC tooling. However, at others the thought of handing over terraform or Cloudformation to a bunch of engineers with no experience in infrastructure(for better or worse) would be a total nightmare. Even with mature guardrails it’s tricky.

From a large organization perspective(Thosands of devs), we've taken both approaches. We give the the ability to do their own thing via tools that we've aligned to, but also provide abstracted options for services like databases. We've been doing this for the last 5ish years and the dev teams almost always choose the abstracted option that we provide them for deploying databases. Some of the value the database team provides is that on the app teams next deployments, they pick up changes to the IAC code... typically security related things or new standards that are being developed). We haven't done services like s3 and dynamodb because they are so easy to set up and deploy, that there wasn't much value in doing that. With the more complicated services like RDS and EKS, we've created abstracted patterns for them to use.

A really interesting topic - I would tend to prefer engineers learn IaC as part of their onboarding. Modules are provided for common use cases like spinning up a database etc. but engineers still have to know how terraform works etc. I currently think having the wrong abstraction is far worse than none - and finding the right abstraction is really hard.

This is a great thread, and it's interesting to see the range of advice from "devs learn IaC" to "encapsulate it all (maybe with Backstage"! I shared some related thoughts on LinkedIn about abstracting IaC through what the Humanitec folks coined as a "Platform Orchestrator" layer: https://www.linkedin.com/posts/danielbryantuk_im-creating-more-content-around-platfor[…]687077158914-ZHVy?utm_source=share&utm_medium=member_desktop

this is a great thread... it would probably be helpful to identify the persona you are when commenting... i.e. app dev vs. platform dev.

We use functions as code generators + temporal for retry persistence, infra as code is too brittle, nearly impossible to test, causes huge issues at scale. See this example docusaurus site, we’ve abstracted building deployments on k8s to a ~6 parameter function call, which is also perfect for AI automation. https://medium.com/p/af999d7ef90a

This is also what we do at Hashicorp for our Managed products.

We find that some engineers want to build with IaC but don’t want to learn terraform. For example, if a network engineer wants to build a mutli region network on AWS with TGW’s, VPC etc. They have to build with IaC for all the usual reasons but just don’t have the time or inclination to write terraform. Abstraction is perfect for this use case. Let the network engineers focus on networking not coding terraform. We’ve written TF modules for exactly this use case (and others) and have enabled customers to roll out their services 80% faster. We maintain the code for them, they build with the code using YAML as their inputs. The builders build, the coders code. We have found with some customers that they start to read and improve the code. So starting off with abstraction has got them up and running on the infra side but access to the code has started them on the IaC journey. This is much better than them struggling to write TF from scratch.