So far most (99%) of the app and infrastructure configuration management setups take a static approach to config mgmt, meaning the setup gets manually scripted by humans for each environment and against a static set of infrastructure components. This static approach causes problems every time a team wants to do things that go beyond the simple update of an image (e.g., rolling back, changing the config, the application architecture, etc.). I highly encourage you to read the article written by @Kaspar about what is a dynamic configuration management and how it could be a life (and time) saving component for your internal developer platform.

still missing the thing thats actually hard about the dynamic environments: data. stateless things can be managed relatively easy in a dynamic fashion still, but when it comes to data, migrations, separating it, datasets, etc, we do not have a good way and this is where it gets hard and actually whats stopping people from doing it. a test environment is not worth much without reliable test data and that’s not simple at all 🙂

I think whether you need completely isolated data sets depends a bit on your use-case, e.g. in our use-case using the same database(s) for all dynamic development environments is mostly fine. Yes it doesn’t cover all use-cases, but most of them for us. In any case there are also tools like https://github.com/postgres-ai/database-lab-engine, which in theory allow to quickly copy your datasets for exactly this use-case, but I haven’t really worked with them yet. Did you?

It’s currently a tradeoff everytime of course and pg is just one of the many examples for having data really. separating into a clean copy is for most of the cases I know a rather hard requirement. fumbling in each others datasets makes things a lot harder to reason about. if your software has sufficient tenancy built in, that is less of an issue as for much code this doesnt apply, but there will always be cases you’d wanna look into it and make sure you don’t get funny side effects. so you’d run dle with humanitec in the cluster basically for the pg usecase?

Yes, DLE was just one example, there are also some providers (e.g. neon) that have branching as a feature in their Postgres offering nowadays. If you don’t want to add yet another thing, you could also hand roll your own schema copying logic. Sadly the obvious restoring a RDS/CloudSQL snapshot isn’t really working as a restore often takes 30mins+ until the instance are accessible :-/

fortunately with tenancy its less of an issue but not many apps are built in a way that allows for testing on the same db without stumbling, which exactly stops many efforts and its usually the apps which can not incorporate yet another external service. this was more or less a hint on “there is need for this” 🙂

👍 it is definitely an underdeveloped area.