Hey Folks, We at Dgraph Labs use Github as our VCS. And we have recently migrated our CI/CD setup to Github Actions. This was a huge win for us internally, especially in a startup setting like ours. Our wins were broadly in these 3 areas

Compute Costs

,

Maintenance Efforts

&

Configuration Time

. With this new setup, we designed & developed

Dynamic AutoScaling of Github Runners

in house. We are thinking of open-sourcing this project. If there is any interest here - pls do reach out. We were able to

_*save ~87%*_ $$

of our

Compute Costs

with this setup. Blog Link => https://www.sudhishkr.com/posts/20230217_dynamic-autoscaling-of-github-runners/

@Sudhish KR this is great. I was exploring something like this a couple of months ago. we eventually went with launching a new ec2 instance but any reason you explored autoscaling rather than launching new ec2 instance each time? the main benefits we saw was that we could customise different ec2 instance for each type of repository

@Srinivas Devaki sorry for the late response, the real problem here was event driven dynamic auto-scaling for our github runners. The event triggers we needed are for multiple repositories (not one). Let me know if you are interested in this, happy to connect and tlak through this.

This is something that’s potentially on our roadmap as well. Have you considered using AWS Fargate? Seems like it could be a good match cost-wise. Although I’m not sure what kind of performance penalties you would get given Fargate startup times.

Hey @Arnas Lukošius - I did explore fargate as an alternative. But the harder problem was not around provisioning a machine. It was more about dynamic selection of machine types based on the load, and getting them up quick. Launchtemplates did the trick in a secure fashion. And was a thin wrapper on top of AWS EC2 - which seems like a simpler solution when compared to the other projects like Philips. I am happy to walk you through our setup, we will be open sourcing this soon.