👋 Hi everyone! I just had a conversation today with a few engineering leaders on building platform teams and a repeated question that came up was, How do you measure success as a platform team, especially when you’re doing a great job and it feels you’re not needed?

Of course it will depend on your context, here I will share what we measured: • Strean-Aligned Teams (SAT for short) Cognitive Load • Lead Time • Failure rate faced by clients Why those 3? Because they were the main purpose when we started the team. I'm sure they will evolve overtime. Lucky us, we were able to measure a good impact on Lead Time and Failure rate on the first quarter. Measuring CL effectively is a little more challenging

Hi Alex, I'd say create some solid KPI trees for engineering productivity, reliability and maybe cost efficiency. The productivity ones will for sure include the DORA metrics, but also more in-depth metrics such a context-switching, cognitive load, build time, developer wait time... If you have multiple platform teams you may wanna assign some focus areas, e.g. for the CI/CD team you can show how your built time or wait time improvements contribute to a reduction in lead time

@Lambros Charissis would you be willing to share your KPI trees? I'm right in the middle of creating them for my teams, and I could use some inspiration 🙂

All of those are good metrics, but I’m also a massive fan of tracking reduction in tickets filed by developers. If your self-service platform is working well, you should see that dramatically reduce over time

So true @Nigel Kersten! We were measuring open tickets by customers as well.

I’m also a fan of doing something that gauges sentiment and happiness amongst the users by using a really simple Net Promoter Score survey at regular intervals https://en.wikipedia.org/wiki/Net_promoter_score

Hi all. Super interesting topic I'm facing the same issue. In our case main purpose is lowering cognitive load of Stream-aligned-teams. But how to measure that? Aside the NPS that is completely subjective, do you have a good approach for this? Thanks a lot

I’d look at internal and external KPIs, one thing that is very interesting to understand how much the platform teams understand the why behind what they are doing as a business objective. A lot of the times the root cause of misalignments isn’t the wrong external KPIs or interfaces built its how well the internal team understands their purpose. I am a huge fan of NPS in reasonable sized orgs for part of the external suite.

^^^^^ those last two points are so important

Thanks so much for all these insights! This is wonderful to read through and one metric that I’m trying to capture is part of DORA metrics and specifically the Deployment Frequency and Lead Time for Changes. Our first full and complete platform team is our Design System Team and they have a direct impact on those metrics as they provide pre-build, ready to use components and modules for product teams. NPS is actually super interesting and will surely try that with our small infra teams, specifically around CI/CD.

@Nigel Kersten in a recent initiative where we migrated from Concourse to GitHub Actions, we tracked the number of CI related questions, and also how many questions we got for each products. Really cool to see our over CI related questions going down as more people moved over to GHA 🙂

As someone who worked on the State of DevOps Reports where we worked with the DORA folks to produce those metrics 🙂 I think they’re fantastic for broad-brush measurement of progress, but I’ve seen way too many teams get obsessed with making minor improvements to them and losing sight of the bigger picture. You should absolutely measure them and look to make big improvements to them, but if you start getting obsessed with fine-grained improvements, take a step back and think about what the bigger picture is with respect to extraneous cognitive load for the people in the system.

Finally got to publishing this: https://medium.com/wise-engineering/platform-engineering-kpis-6a3215f0ee14