Hey guys. I would like to spin virtual place in th...
# general
Hey guys. I would like to spin virtual place in the cloud for each of users. This virtual place should be able to handle kafka instance, postgres and http/ws services. Currently I have CRD that I replicate with terraform. I would like to craft additional features (like scaling to zero if no usage, scaling up if traffic, respawning machines from zero etc). Can your recommend any way to automate it in scale?
You would need to apply a golden path and self service concepts.
You would need an automation (CI/CD pipeline or a page on IDP) that allows users to create infra from preimplemented templates.
Then user should be able to remove it, recreate it, scale it up and down.
All that is created should be placed in some kind of infra catalog, which contains information who owns what, when was it created, etc.
Things like scaling up and down depending on traffic would usually be baked in in each of the templates
for each technology separately. If you are using k8s it is easy to setup Keda autoscaling based on multiple metrics like http traffic but also numer of messages in queue/topic in kafka
With Infrastructure catalog you could also iterate through your resources and shut them down and spin them up at certain hours, or depending on metrics
The bigger the scale the more difficult the task.
If you have 200 devs using that, the costs will add up quickly, so you need to be able to say who pays for what 😅
Good luck 🙂
You could spin up vclusters for this: https://www.vcluster.com/pro/