I'd love to hear your feedback from my thinking here: https://twitter.com/raquelpau/status/1575912572452507649

Gitlab has the concept of templates which you can import into your ci file on each repo and then add on top of that, so the main template can be updated without having to go to each other repo and make commits - does that help if im reading it right?

@Hugo Pinheiro I do not think so. With GitLab, (1) this approach means that you can impact to downstream repositories if your changes do not work because there is no PR flow; and (2) importing templates still requires a manual intervention - you need to update and integrate the template in an existing pipeline. My view is that even new templates should be provided with automatic PRs.

you are absolutely right @Raquel Pau Fernandez - when i was running a platform for data engineers I build a system to watch the users repositories for changes to a file called

deploy.json

. Based on that a CI pipeline would be created from a template. It avoided having the CI pipeline live in the application repo and it vastly constrained what those developers could do. Guidance we wrote is here: https://user-guidance.services.alpha.mojanalytics.xyz/build-deploy.html#deploy-json

exactly, that is my PoV, we need a way to combine customizations with Golden Paths that allows to maintain those Golden Path updated

I join @Hugo Pinheiro as I find his solutions to be a good base to build on. For your downstream repository updates I will go with something like dependabot or renovate, which will detect new tag version on the gitlab-ci template repo and create a Merge Request in the repositories that depends on it (dev repositories). It’s a solution close to what @Ravi Kotecha proposed, but with existing open source tools. It will of course need a bit of configuration: which repos to listen, how many times a day to run this, etc.. This way you have • centralized product to focus on : the gitlab-ci template repository • automatic delivery: renovate (or dependabot) Getting started with this for dev is : 3 line of code in a

.gitlab-ci.yml

file to include your template and simple enough (empty ?)

renovate.json

file to get automatic update on this template. Edit: you already mention dependabot in your thread 🤦‍♂️

I think we can, dependabot did it. The problem is that you can't have huge building blocks (besides using dependabout) that do not accept hook logic or customize them in downstream repos. This is why I think that the current model is broken because it is designed to be performant starting fast (hello worlds), but without applying OO /reusability principles - There are just functions, and you can only extract logic

Indeed, I’m not a dev so maybe I’m wrong, but it feels like we only have inheritence in pipelines solutions but we should have composition as you stated Also, I assume that the audience of your thread could be Ops oriented people (people that create and maintain CI/CD tools are more often SREs), explaining open-close principle and Golden Paths in your thread would have been great I think ! 🙂

Thanks for the feedback @antoine meynard. You gave me the idea to attach more tweets 😛

Wow! At Netflix we came across this a few years ago. I spearheaded a project there for this exact problem: http://managed.delivery which is now the default approach for all new apps there. Exciting to see this concept becoming more widespread!

Michael, the project does not look very active. Could we have a session together to learn from your experience?

Sure! I left Netflix last year. I know they moved away from public / oss contributions. So that’s probably why it’s not active.