Hey all, has anyone attempted to provision production level infrastructure with Terraform CDK yet?

We had some luck doing this, but didn’t get very far ultimately when leadership shifted what was important to them about three months into the project. I was/am a big fan of it. We ended up creating a simple python service in front of cdktf, storing the configuration data for “stacks”, then performing an async tf apply on an ephemeral image. Our main objective was self-service infra so that users “didn’t have to learn terraform”. We got all the way to production with RDS, creating subnets, param groups, instances, databases, secrets (in vault), users, etc.. Happy to chat more and answer any additional questions, if you have them. Since our goal was self-service w/out TF, our plan was not to expose cdk directly, but a RESTful service instead.

That sounds excellent, Might take you up on the offer of a chat on the subject. Did you find any issues with the CDK that you could see could make you wary?

The only thing that was weird was that it was essentially procedurally generated code that required, this is my memory so might be incorrect, node to execute some of the code outside of the main process. So we didn’t experience any direct issues with that order of events, but we were wary and kept an eye on it, as debugging and troubleshooting could potentially be difficult. I liked how external/third-party providers could be integrated; we packaged these as pip-installable packages (e.g. vault). For budgets, we did a few things, but we’re trying to tackle this more holistically so it wasn’t a direct concern of our service: • We had lambdas that cleaned up our tests if we had failures • We utilize cloud custodian for scheduled and triggered alerts + cleanup ◦ RDS instances in develop environments with no connections over the past N weeks are deleted ◦ Stale EBS volumes ◦ etc • We additionally use cloudability + backstage for monitoring team-specific spend with alerts coming from there • Finally, we abstracted some of the weird sizing of instances and instance types to small, medium, large, and xlarge; so we could enforce the use of specific DB types, versions, sizing, and that users used instances for which we purchased RIs We could enforce certain labels and tags at various layers too, which was nice. AWS org policies and service schema… you don’t have to use our service, but we’ll make sure you meet our org standards by default

I really like that approach, Thank you for sharing with me.

I’m sorry, I’m not sure I understand the question?

Inplace of the self-service environments you would provision via the work you did, sorry. I (might of wrongly) assumed it was so developers could provision environments to work with.

Ah I think I understand…. please let me know if this is off mark: We have an internal wrapper around Terraform, which developers currently use. They can also provision in dev environments in AWS manually, though we discourage that. So the problem statement still remains true and unsolved, but there are ways developers can “self serve” the creation of resources in our AWS accounts/environments.

Right ok, that makes sense! sorry for the confusion!

@Rory Scott - was there any specific reason you chose terraform CDK over pulumi?

Terraform CDK looks nice. The best thing about it is that it can generate Python/Go/etc code for all Terraform modules, which are about 1000+? I was using it only for a while with Gitlab, Kubernetes and AWS provider as far as I remember. The pain was a very long and memory consuming classes generation for larger modules like AWS, but If it could be done once and then pushed to private PIP repository then probably this issue can be worked around - a CI/CD pipeline could be done for this 🙂

@Ram Kumar G We are a big org with a lot of decisions that have been made, so we were trying to rock the boat as gently as possible simple smile There are a few underlying reasons outside of the political. Primary problem was that developers didn’t want or didn’t have time to learn “terraform”, which in our company means cloud infrastructure. They all know how to hit a RESTful endpoint though. Next, we have a large investment in terraform already, including modules and third-party providers; so instead of injecting a brand new tool, we decided to try and provide a pierceable abstraction into our existing IaC platform. You could write terraform, use our existing modules, etc. or you could use a simple form, which used our modules in the end anyway. Pulumi’s experience was also not as nice, from a developer perspective, when we examined it. We did look at pulumi and just using cloud SDKs, but for the reasons specified above, we went with cdktf. I really like how you can render terraform json from a configuration, making it easy to integrate with a simple terraform plan or apply, allowing your execution model to happen the same regardless of if someone wants to write terraform themselves from scratch, use modules, or use our abstraction.

I was surprised that the regular Terraform docs are almost 1 to 1 valid with generated Python code (difference is e.g. in camelCase vs snake_case), thats a for sure big advantage of CDKTF.

We had ci pipelines to generate the provider sdks…. I’m not 100% sure how they did it, but they used the tf modules as source, so makes sense it was nearly 1:1. Was definitely nice once you wrapped your head around it, made writing non-hcl code really straight-forward.

build:
	cdktf get --language=python
	mv src/vault src/cdktf_vault
	python3 -m pip install --upgrade build
	python3 -m build