https://platformengineering.org logo
#platform-design
Title
# platform-design
r

Raquel Pau Fernandez

09/30/2022, 8:00 PM
I'd love to hear your feedback from my thinking here: https://twitter.com/raquelpau/status/1575912572452507649
h

Hugo Pinheiro

09/30/2022, 11:34 PM
Gitlab has the concept of templates which you can import into your ci file on each repo and then add on top of that, so the main template can be updated without having to go to each other repo and make commits - does that help if im reading it right?
r

Raquel Pau Fernandez

10/03/2022, 8:55 AM
@Hugo Pinheiro I do not think so. With GitLab, (1) this approach means that you can impact to downstream repositories if your changes do not work because there is no PR flow; and (2) importing templates still requires a manual intervention - you need to update and integrate the template in an existing pipeline. My view is that even new templates should be provided with automatic PRs.
r

Ravi Kotecha

10/03/2022, 8:57 AM
you are absolutely right @Raquel Pau Fernandez - when i was running a platform for data engineers I build a system to watch the users repositories for changes to a file called
deploy.json
. Based on that a CI pipeline would be created from a template. It avoided having the CI pipeline live in the application repo and it vastly constrained what those developers could do. Guidance we wrote is here: https://user-guidance.services.alpha.mojanalytics.xyz/build-deploy.html#deploy-json
r

Raquel Pau Fernandez

10/03/2022, 9:00 AM
exactly, that is my PoV, we need a way to combine customizations with Golden Paths that allows to maintain those Golden Path updated
a

antoine meynard

10/03/2022, 10:29 AM
I join @Hugo Pinheiro as I find his solutions to be a good base to build on. For your downstream repository updates I will go with something like dependabot or renovate, which will detect new tag version on the gitlab-ci template repo and create a Merge Request in the repositories that depends on it (dev repositories). It’s a solution close to what @Ravi Kotecha proposed, but with existing open source tools. It will of course need a bit of configuration: which repos to listen, how many times a day to run this, etc.. This way you havecentralized product to focus on : the gitlab-ci template repositoryautomatic delivery: renovate (or dependabot) Getting started with this for dev is : 3 line of code in a
.gitlab-ci.yml
file to include your template and simple enough (empty ?)
renovate.json
file to get automatic update on this template.
Edit: you already mention dependabot in your thread 🤦‍♂️
+1 on the need for pipelines solution that apply open-close principle, but for the adoption problem I think we will never solve communication issues with more complex tools 😅
r

Raquel Pau Fernandez

10/03/2022, 10:41 AM
I think we can, dependabot did it. The problem is that you can't have huge building blocks (besides using dependabout) that do not accept hook logic or customize them in downstream repos. This is why I think that the current model is broken because it is designed to be performant starting fast (hello worlds), but without applying OO /reusability principles - There are just functions, and you can only extract logic
a

antoine meynard

10/03/2022, 10:48 AM
Indeed, I’m not a dev so maybe I’m wrong, but it feels like we only have inheritence in pipelines solutions but we should have composition as you stated Also, I assume that the audience of your thread could be Ops oriented people (people that create and maintain CI/CD tools are more often SREs), explaining open-close principle and Golden Paths in your thread would have been great I think ! 🙂
r

Raquel Pau Fernandez

10/03/2022, 10:50 AM
Thanks for the feedback @antoine meynard. You gave me the idea to attach more tweets 😛
m

Michael Galloway

10/07/2022, 1:10 AM
Wow! At Netflix we came across this a few years ago. I spearheaded a project there for this exact problem: http://managed.delivery which is now the default approach for all new apps there. Exciting to see this concept becoming more widespread!
r

Raquel Pau Fernandez

10/10/2022, 12:38 PM
Michael, the project does not look very active. Could we have a session together to learn from your experience?
m

Michael Galloway

10/11/2022, 12:13 AM
Sure! I left Netflix last year. I know they moved away from public / oss contributions. So that’s probably why it’s not active.
22 Views