This message was deleted.

Hi @Jakub Maziarz, it’s indeed a broad topic. I can even write a 2000 words article on this one 🙂 (cc @Morgan Perry). Do you have any specific questions? What are your main concerns?

Appreciate swift response @Romaric Philogène Please do write an article 🙂 What I'm looking for is a context for basic discussion of IDP vs. PaaS. I imagine many organizations start with trying to understand how both would work for them (and which one to choose). I'm struggling to find comprehensive overview in that topic to go deeper. It buggs me that most of "build vs buy" articles are repeating the same arguments on superiority of IDP for Enterprises, so I thought question on limitations of IDP would help me get more substance to think about. One great discussion was here

The discussion you mentioned is a good start. There are a bunch of articles inside this thread you can take a look. Basically, IDP is a generalization of what a PaaS is. I think it could also be an excellent article to write - what’s the difference between a PaaS and an IDP.

Basically, IDP is a generalization of what a PaaS is

This statement surprises me 👀 So I second it being interesting to hear more from an article! The reason it surprises me is because I think of an purpose built internal platform as a more specialised version of a PaaS. A recent analogy I have been using is to say that a purchased PaaS is the team topologies “as-a-Service” pattern without first the collaboration between the service builders and users. As in, no one from Heroku or Cloud Foundry or other PaaS companies has come in and collaborated with your company’s delivery teams on what they need. They built something really powerful but also really general to the market needs. In contrast, an internally built platform can (and should) be built from deep collaboration on business needs. Thereby making it completely custom to the org though likely relying on a lot of purchased PaaS options behind the scenes.

I agreed with you @Abby Bangser 💯 What I meant was on the technical side, an IDP fits better your technical environment and constraint than a PaaS. That’s what I meant by “generalization”

I believe that the core of the question boils down to “what problem are you trying to solve?” and “how much are you able to invest?” I read somewhere that an MVP for an IDP can be an intranet page, or a list of APIs. Probably if you have 2.500 microservices and 15 engineering teams, your challenges are different than if you are a scale-up with 25 engineers in the house. My take to the question is: Start by asking your engineers what prevents from being more efficient and deliver value faster and then map it to build vs buy.

what are the constraints and limitations of building your own platform?

To your original question Jakob, I think there are a few things I have run into when I have done this… 1. Coverage across the platform needs. Often because building takes a long time I have ended up with some offerings on the “new platform” but a lot of things not yet there. This creates more havoc because people don’t know where to look and the “legacy” stuff (read the stuff that actually makes the company money!) is getting even less love. 2. API design. I have often tried to start with really basic APIs like “just give me the database size and we will do the rest!” but slowly but surely have ended up recreating the APIs that I am depending on for implementation (e.g. the Terraform module fields or the Helm chart fields). I think this is because I have struggled to create a true client side / server side split with my implementations so there has been major boundary context leakage. 3. Avoiding Not Invented Here (NIH) syndrome. Building is fun. When building people tend to want to build moar despite the value in leveraging other tools within the building process (e.g. use EKSCTL instead of trying to recreate how to build an EKS cluster or relying on a Terraform module in lieu of building something yourself). But this has not only been an issue on first build, it is also on continued refinement. In the case where the company has purpose built something out of necessity, but then a tool comes along and can do it for you, the team needs to want to evolve to the external solution rather than preserve their own tool.

The PaaS approach is to “take it or leave it”. There is no way to customize it or at least integrate it properly into an existing environment.

Agreed. The only option is to build around the PaaS in a way that customizes it. In comes the crazy amounts of bash and other scripting which can be hard work and hard to maintain.

@Romaric Philogène @Abby Bangser I agree with you. But I would also like to add, in my humble opinion, certain nuances to these remarks. First of all, the technical stack. Indeed, with the advent of different types of services (cloud, SaaS, OpenSource, etc.) and potential multitudes of languages, a technical stack can quickly become heterogeneous and PaaS may no longer be suitable (multi PaaS? multi tools?). For me, we have to see the IDP as a whole and not just the possibility of creating a web page or an API (I'm schematizing here deliberately 😝). It allows total control of all of its assets by allowing the work of technical teams to be “PaaSified”. And I would just add, so as not to go on, that we must never forget the security layer as well as data governance. Reducing IDP to a "controlled PaaS" for me is too narrow. It is a model architecture between PaaS and IaaS (maybe a word already exists for that 🤷‍♂️). I also think there are 2 types of IDP. The first under batteries (or under steroids ^^) which do most of the work (maintenance, upgrade, networking, vpc, etc.) and allow you to add your own application layer and logic, and others which allow you to create your own workflows , its own "low levels", ... and to create it entirely from A to Z. It is a question of choice, of means but above all, and I am convinced of it, of necessity. In short, it's just my 2 cent feeling ^^

Thank you everyone! I'm blown away by responses. You all convinced me it's a topic worthy of a deep article, already started sketching one and will gladly share & invite to collaboration on this (e.g. by doing wider survey on the topic). Personally, I'm not a fan of definitive statements X is best, Y not. And I don't aim to answer that debate with the article, because to me it depends, as no organization is the same. I'm looking to provide more comprehensive starting overview of both IDP vs. PaaS whenever you face that decision, to help build your shortlist. Again, any feedback, comments, ideas are very welcome.

@Mickaël Gentil I really like your statement

The first under batteries (or under steroids ^^) which do most of the work (maintenance, upgrade, networking, vpc, etc.) and allow you to add your own application layer and logic, and others which allow you to create your own workflows , its own "low levels", ... and to create it entirely from A to Z.

👏

Thx @Romaric Philogène and We have choosen batteries to keep KISS principle 😁

I think I know this product 😂

A bunch of this is dependent if you start from Prod -> Dev or Dev -> Prod. Its very hard to try and meet in the middle in my experience. Additionally there is substantial complexity if you have legacy systems and dependencies. Complexity comes from lack of first principles thinking when painting a north star. Technical complexity / sophistication works out fine if the principles that you used to arrive at whatever architecture are clear. I would encourage you to avoid anything about technical solutions till you understand the problem statement and organizational principles

agree with @Abby Bangser Based on several platform deployments I've conducted for various clients operating in different business domains, one rule has consistently held true: there's no one-size-fits-all IDP, that meets the requirements of everyone. Every platform project should start with a solid workshop aimed at understanding how development teams work and how they support business streams. Often in our workshops and subsequent architecture work, we aim to model the platform's architecture as a set of services, which by definition should be tailored to the organization and the challenges it faces. This approach recognizes that certain services may repeat, while others will be specific to a particular organization or industry. There are also positive aspects to beginning a platform project. You don't necessarily have to revolutionize the entire tech stack of organization. Many organizations have already adopted DevOps practices and related technologies. In such cases, it's crucial to thoroughly understand the organization's needs and the IT department's needs to accurately define the service layer that will address the platform's customers' needs. In many cases, services can be built on technologies with which the DevOps or Cloud teams are already familiar and comfortable. Another outcome from my previous thougths is -> if the platform should be tailored to organization profile, what is the right size of organization for the IDP project to be worthwhile? 🤔

if the platform should be tailored to organization profile, what is the right size of organization for the IDP project to be worthwhile? 🤔

Absolutely a consideration! As for an answer, well “it depends” sucks but is kinda does. In part because I find there are qualifiers to size such as: • Risk profile (a small but highly regulated company may need one sooner than a large but mainly “fun” domain company) • Scaling ambitions (small but looking to double in the next quarter may want standardisation for onboarding) • Engineering capacity / skill (a highly experienced team may be able to get past IDP face value quicker and therefore it is worth investing sooner) • …

Having built such a platform from the ground up, I think the paths to developer self-service really eventually converge to “creating your own opinionated building blocks and abstraction layers”, especially in larger organizations (I’ve only been in large ones). Sure it’d be great if you can find LEGO blocks out of the box and they “just work”, but from my experience they never do (my samples are all from highly-regulated industries) - or if they do, they are way too abstract and serves no good use for the developers. The concept of platform engineering or an IDP product to me represents an advocated pattern/workflow (in the form of an orchestrator, for example) and some mechanisms to help you get there quicker.