:thread: CI/CD & build automation
# platform-culture
e
🧵 CI/CD & build automation
Hi folks, the poll didn’t get a ton of votes, but the few it did get were consistent with the anecdotal findings I’ve been seeing: dedicated CI/CD team seems to be getting increasingly uncommon, instead being owned by either SRE / Platform Eng or some part of the core eng team. There are a few other emerging trends that are interesting: - more & more security tools integrated into the CI pipeline, and require more babysitting or even manual operation per build (security is of course very important.. but so is delivery velocity) - in the case where its handled out of core eng, CI can get fragmented across teams, requiring “CI for CI” or a manual approach to stitch everything together - deployment no longer goes to prod for various reasons, but to staging (which is not exactly like prod), and there are a set of poorly documented things that must be done to take a build from staging to prod, manually the term “CI/CD” itself seems to be falling out of favor, with some folks preferring to call it things like “build automation,” possibly because it’s not longer fully Continuous I’d love to hear any thoughts, opinions, personal experiences folks here would like to share! (PS if anyone feels there is a better channel for this, pls advise)
a
Probably bc ci/cd aren’t serving the same business goal https://www.prodvana.io/blog/ci-cd-joined-forever-never-the-same
a
CI and CD are basically solved problems that there’s dozens of off the shelf tools that will support it for you. And also with a small sample size you’re probably missing large enterprises who haven’t figured that out yet.
a
Neither are really solved problems 😉 The market is extremely fragmented w/ toolchains that breakdown in cloud native worlds
Or another way of saying that is “The problem has changed” in 2023
a
¯\_(ツ)_/¯
Those teams were a reaction to how much yak hair you needed to shave to do them effectively 5 years ago. That is simply not the case anymore.
a
I think it depends on your definition of CI/CD. There are quite a few more business processes that are involved in getting a deployment to production. For example if you’re working on AI software / LLMs workload migrations, capacity alignment etc becomes part of your delivery process, testing and CI is also quite different in that space. If you’re dealing w/ storage / data systems long running delivery is the norm and this means you probably want to think about how pipelines (bleh) get re-run or don’t get re-run. There’s quite a bit that has been changing.
If you’re a Web2.0 CRUD app sure, CI/CD is pretty static and boring now. But if you’re looking at the next wave of applications that are being built its quite different. (we’re not even talking about the serverless type architectures and how to weave them together)
(there’s also the SSC type work that has to be done that was abysmal 5 years ago)
e
Andrew, I assume you’re referring to the NIST framework? I agree, that continues to be a major source of added complexity in the dev cycle. It seems to be gaining traction in more industries too, as people try to figure out a way to finally stem the tide of never ending security issues. OTOH, adding slow, semi-fragile scanning and analysis tools into the pipeline is hampering the original intent
work is still going on there today, NIST is collecting comments on the latest draft through next month
That article is great BTW! I will need to read it more detail later on, but even at a surface glance it touches on some key points. CD is great for consumer apps, where no one user has any say in what happens. but for enterprise, it was never a good fit. Enterprise customers are not generally interested in having the software they use change at random intervals. It also doesn’t fit well at the “edge” for non-cloud compute platforms, which have a totally different set of constraints.
(talking about CD all the way to prod here, as opposed to an internal staging env)
a
yup that’s what we built :) the problem in cd is it’s a pipeline and not a convergence engine
we solve the enterprise and the edge problem
e
interesting! can you share the company / product?
a
e
oh haha, you work there! 🙂
a
we have a bit of experience with this ;) we owned the dropbox installer and all infra there
so we saw this use case entirely
it’s exactly what you described as the problem statement
e
very cool! I will look more into this. thank you!
m
I am not quite sure about the intended definitions of CI/CD in this post. Both are not primarily about pipelines or tools, they rather are practices, which ultimately require automated pipelines. The original definition also never intended for separate CI/CD teams, they rather defined self-sufficient, cross-functional teams as a starting point. Continuous Integration: Integrating to a shared main branch at least once a day. Continuous Delivery: A set of practices for delivering better software faster by effectively doing TDD and ATD, pair programming, trunk-based development etc. and building security and other compliance related requirements into the software and the resulting pipelines which automatically checks for everything. Dave Farley also defined CD as scaled XP.
e
Hi Murat, thanks for adding your perspective! I assume you’re referring to the writings of Martin Fowler, Jez Humble, and their collaborators?
m
Hi Eric exactly! The primary purposes and definitions are described in the main sources/books and they have gotten blurried after several years. That's why I am a proponent of referring back to them and seeing that it is the principles and practices which need to be understood to be able to fully utilize our beautiful tools of today. I am mainly referring to following books: extreme Programming, Continuous Delivery, Accelerate, Continous Integration etc.
a
☝️ This is much better way of saying what I was trying to get at.