hi team, I struggle to connect IDP with the platfo...
# product-management
g
hi team, I struggle to connect IDP with the platform concept defined here https://www.amazon.com/Platform-Revolution-Networked-Markets-Transforming/dp/0393249131 It's a great read that explains how companies like Amazon's online store, Google's search engine, Meta's Facebook (and obviously others) deliver multisided platforms where content is created by 3rd party suppliers and consumed by users. Essentially the power of these platforms lies in community and interactions that take place within the community of users and suppliers (and other sides), while Google, Amazon, Meta 'just' deliver the platform mechanisms and reduce blockers for the interactions to take place and increase over time. so how does the 'platform' concept above apply to the 'Platform' part of IDP? IDP is delivered by the platform engineers(company employees) and 3rd party solutions and then it's consumed by users (other company employees). Platform engineers build platform (like Meta employees build Facebook) so they do not really form the community. On the other hand, there are other employees who consume the platform - this is one side of the platform, the users' side, but what about the suppliers' side? Shouldn't the platform offer capabilities which enable some of the users to become suppliers and build components (not the entire CI component but ie. GitHub action for Sonar integration) that can be used by other users? This makes the entire community contribute to the platform's growth rather than just the platform engineer team. Any thoughts?
l
I have read that book, it’s good - but is not related to platform engineering at all.
So how does the ‘platform’ concept above apply to the ‘Platform’ part of IDP?
These are two entirely different concepts based on the same name. Platform in engineering is about enabling teams, it’s an internal structure and approach. Platform business models for multi-sided markets are a different thing. I’d not confuse the two, too much. What are you trying to explore and/or achieve?
g
I wondered if there's a way to mix these ideas and get combined benefits
a
Interesting perspective! Let's explore how we can incorporate the concepts of supplier and consumer sides within Identity and Access Management Platforms (IDPs). In the context of IDPs, we can consider the following roles: 1. Consumers: In this scenario, consumers are the internal product development teams within the organization. They seek to leverage the IDP platform to streamline their workflow by having immediate access to essential components, infrastructure, security measures, observability features, and more. Essentially, they aim to ensure that all the necessary elements are readily available from day one, enabling them to focus on their specific product development tasks. 2. Suppliers: The supplier role is fulfilled by an internal platform engineering team. This team operates in alignment with the organization's standards and long-term vision. They are responsible for crafting various templates, configurations, and resources that serve as building blocks for product development teams. These templates are designed to expedite the initial setup process, ensuring that product teams don't have to start entirely from scratch. Instead, they can rely on these pre-established foundations to gain a significant head start in their projects. This approach allows for a symbiotic relationship within the organization: • Consumers (Product Development Teams) benefit from reduced development overhead and accelerated project kick-off thanks to the preconfigured components and templates provided by the internal platform engineering team. • Suppliers (Platform Engineering Team) play a pivotal role in shaping and maintaining the IDP platform. They ensure that the templates and configurations align with the organization's standards and future objectives, ultimately fostering efficiency and consistency across product development initiatives. By emphasizing the supplier's role, you're suggesting that the internal platform engineering team should continuously enhance and expand the set of available templates and configurations, responding to evolving needs and technologies within the organization. This approach empowers the entire organization by leveraging the expertise of the platform engineering team and facilitating a smoother and more efficient product development process.
g
Anshul, great summary of IDP roles through the lens of Platform Revolution ideas. My idea is that Consumers and Suppliers overlap to some extent. Let's compare it to Amazon store. You can come to Amazon for most of the time as a buyer(consumer, default role) and buy items on the platform but at the same time, you may want to sell something occasionally. There's also a flipped scenario in which you're a seller (default role) but sometimes buy something from Amazon. In the company I work, within Consumer segment two subsegments clearly emerge: 1. skilled consumers, who understand what is CICD process about, and why it exists, they can adopt IDP components to the needs of their team, maintain this integration as well as the component itself (if the component is on-prem rather than SaaS), they are very oriented in the technology stack behind IDP components, can use them for their advantage and what's actually the most important they are looking for ways to maximize automation and DORA metrics, they are heading towards DORA Elite 2. Unskilled consumers, who are purely focused on business application development and have no interest in DevOps, IDP integration and components, just want their build to be a green and appropriate level of debug info in case of build issues. If the CICD flow works, let it work, don't touch, don't improve Obviously, there's a deep issue with #2 segment which nails down to work culture, but I want to focus on #1 segment. Let's use this scenario as an example. The developer pushes the code to the repository and there's a series of commit validation hooks like branch format, message format, existence of some specific identifiers like user story or bug ID in the message etc. Such hooks could be delivered by the PE team but this work will depend on PE team priorities. If this work is less relevant than other items in the backlog it's guaranteed to never happen. However, there are teams that need these hooks and their productivity would benefit if only hooks are in place. So my suggestion is to allow consumers (who are skilled enough) to build a hook that could be used also by other teams. However, I don't mean to simply give a consumer permission to extend IDP capabilities, I mean a platform (in Platform Revolution terms) which enables the delivery of such hooks in an easy manner like Android apps are delivered to Play Store by 3rd party developers. With my initial post, I wondered if IDP architecture and processes could enable a community of internal users (beyond the PE team) to extend platform capabilities (users who have appropriate skills and are willing to do so). I referred to the Platform Revolution book because it describes the supplier and consumer model with the platform being an orchestrator of consumer-supplier interactions and I believe this model could be implemented within IDP.
a
Hmm @Grzegorz Wojciechowski I believe I understand your concept. You're essentially looking for a plugin feature or a similar mechanism that consumer teams can develop independently. This approach is reminiscent of Backstage, where various plugins can be created and utilized as needed by different teams. I've encountered a tool with some similarities, and I'm currently exploring its capabilities. It appears that this tool might align with your scenario. For more insights on this topic, I suggest consulting with @Abby Bangser, who can provide further information, especially in relation to Kratix.
l
With my initial post, I wondered if IDP architecture and processes could enable a community of internal users (beyond the PE team) to extend platform capabilities (users who have appropriate skills and are willing to do so).
Hrm, this conversation now kind of reminds me of inner source culture, e.g. the gig marketplace pattern. There’s a set of patterns in this online book that might be inspriation.
g
@Anshul Garg Backstage probably could address some use cases but really what @Lou Bichard referenced to, hit my point. So let me rephrase the initial post. I wondered if and how IDP architecture and process should support inner source culture?
a
hmm interesting !! Even I wasn't sure of Inner-source Pattern !! So, can you let me know how this pattern might help to address this use-case. Would like to dig deeper into it.
a
These are two entirely different concepts based on the same name. Platform in engineering is about enabling teams, it’s an internal structure and approach.
I think we already have a lot of examples in this thread as to how we could enable this in internal platforms but many implementations aren’t there yet. Thank you to @Anshul Garg for the shout out because he is exactly right that this ecosystem is one of the big deliverables that can be achieved with a platform framework like what we are building with Kratix.io Frameworks create a way to enable the three sides of platform ecosystems that you mentioned. Consistent interfaces for users, plugin points for 3rd parties and ease of creation when it comes to builders.
I wondered if IDP architecture and processes could enable a community of internal users (beyond the PE team) to extend platform capabilities (users who have appropriate skills and are willing to do so)
Yes, I hope so! We are building Kratix to be a tool implementation to support the ecosystem and interactions models defined by Team Topologies. To not only enable inner sourcing in concept (PR culture etc) but through purpose built features / hook in points. Our key domain object is a Promise which is an encapsulation of how to deliver anything “as a service”. These are built to be composable and extensible. E.g. you may have a Postgres Promise that can then be independently requested or provided as a part of the larger “demo app” Promise or even larger “production environment” Promise. You can also have a more “advanced” version where you enable access to all the bells and whistles or extend that to abstract it into an “easy” version of postgres where you just enable configuration of size. Access to these Promises can be widely available or controlled through RBAC (e.g. you may only enable access to the “advanced” versions after verifying a certain level of skill by the requesting users. Within a Promise there is an idea of Workflows, basically the actions executed when customising, deploying and managing that service. These actions are currently containers so are independently composable and extensible. For example you may want to codify PCI compliance reporting or cost analysis into their own single containers which then can be included in many Promises. We are currently working with organisations where their specialists are generating either workflow containers or complete Promises which can then enable the entire organisation to leverage their knowledge / specialty without asking them to learn it all.
I realise that last bit gets a bit more into the details of our specific framework. Happy to chat more on that. But I do hope the biggest takeaway is that this concept of thinking about platforms as more than just consumers and producers, but also an ecosystem is extremely relevant to internal platforms at scale and can (and should) be enabled by platform tooling!
p
In my opinion, the platform concept does not differ whether external or internal. Amazon, Google, Facebook provide a service that is created once and utilised by many in different ways to create some value. An internal platform offering provides a service created once and used by many engineering users (whether software, data or otherwise) to deliver value. Utilising a community to scale up the platform offering and create networking-effects (see wikipedia https://en.wikipedia.org/wiki/Network_effect) can be applied in an internal platform (inner-sourcing is indeed a great approach for this). There is no 11th commandment stating that an IDP must be delivered only by the platform team(s) 😉
g
@Pavel Goultiaev networking effects is one of many well-defined mechanisms used by Platform Revolution and it's intention is scaling the platform up, it is described and it just need to be followed. And there are other mechanisms as well, that are researched, validated with real use cases, described and proven to work. Kratix.io brought by @Abby Bangser is a specific implementation of some Platform Revolution concepts in IDPs, this exactly the type of solutions I meant when I started the thread. Now, using Kratix offering as an example how to connect it back into architecture and principles of IDP and platform engineering? In other words which IDP's, PE or Team Topologies principles are design drivers for frameworks like Kratix.io? This kind of phenomenon happened when product management principles has been applied to IDPs and Platform as a Product emerged which resulted in defining business model for Platform Engineering: 1. Customer segments as internal company employees 2. Value proposition as engineers productivity improvement 3. Measures as DORA metrics 4. Customer relationships as Team Topologies interaction modes 5. Solution as IDP 6. Channels, key activities and so on But there's more next to business model: platform product manager role, design thinking, UX and DevX, user research, hypotheses driven development and so on because essentially IDP is a product so it is govern by product managment rules. I feel it's the same thing about IDP and Platform Revolution ideas which can boost IDPs even further. I did not find any documented attempts that would link these two therefore started the thread. I don't think we can solve it here if it has not been tackled before but I'm glad this sparked the discussion and many of you get involved.
a
I agree this has been a really interesting thread to get started, thanks for that! I also agree that I have seen a lot of reference to Team Topologies or Platform as a Product but not seen as concrete connections to the tools in the market. But then again, our marketing is terrible so likely I am also missing it in other people’s webpages / docs! In response to…
In other words which IDP’s, PE or Team Topologies principles are design drivers for frameworks like Kratix.io?
I think I mentioned earlier, we are fundamentally driven by Platform as a Product, Platform Engineering as a core competency and Team Topologies. But how? • Platform as a Product: Products need to be evolved and personalised. A framework can/should enable an organisation to build API first enabling refinement and diversity of interfaces (customers are already using backstage, port, custom UIs, simple yaml files, and slack). In addition, it is built with composability in mind, enabling stacking of low level implementations into higher level abstractions, sometimes simply removing some of the options to enforce company defaults. • Platform Engineering as a core competency: So many of IDP related products today are focused on application developer experiences (or end user experiences). We need more tools and frameworks for platform builders and focuses on the experience of building platform offerings which we believe in turn enables those platform builders to iterate on and build the best user experiences for their context • Team Topologies principles: If TT is about leveraging different team focuses and inter-team interaction patterns to enable faster delivery flow. Then what we need from our tools is to support the evolution of offerings from collaboration through to “as a service” scaling (where necessary) while enabling enabling and complicated subsystem teams to provide their offerings where the stream aligned teams can most benefit from them. Hopefully that helps to understand what I am thinking, and that clearly influences Kratix, but honestly, I want to hear who else is building tools with these (and other!) principles in mind that focus on the product, the builders and the ecosystem around internal tooling.
l
A framework can/should enable an organisation to build API first
Generally, I agree. Not sure if API is the only abstraction that’s useful / needed. For instance templates and patterns are a big part here, which maybe aren’t considered APIs in the traditional sense, but are certainly abstractions. Very few tools out there enable the platform engineering model right now. Most tools in platform are shifted right. IDP, backstage, most security tools are focussed on ingesting existing data and systems, not bringing collaboration forward.
So many of IDP related products today are focused on application developer experiences (or end user experiences). We need more tools and frameworks for platform builders
Strongly agree. This was the final note in my discussion with @Saim Safdar yesterday talking about platform as a product.
If TT is about leveraging different team focuses and inter-team interaction patterns to enable faster delivery flow. Then what we need from our tools is to support the evolution of offerings from collaboration through to “as a service”
I believe another industry anti-pattern is the obsession with making multiple disciplines (devops, devsecops…) as the same job roles as the only way to find and foster collaboration between areas. This is not the case. I believe tools should embrace different personas and teams and then enable them with interfaces and tools that make sense for their individual piece of the puzzle.