@Lambros Charissis thanks for your talk on how to measure platform engineering impact. In my experience one of the most challenging aspects of platform engineering is sustaining investments to continue and grow. I would love to get insights in how you came to the $cost for some of these metrics. I understand the high level and understand for such a short talk it is beyond scope to explain it, but it is the real meat of the story. I have worked with many of the same metrics and calculations but have always lacked the right material to defend it when put against projects investments where ROI is easier to calculate such as end products sold to external customers. E.g. You mentioned to for example take cost of a feature as an aspect to calculate impact of an engineer/hour. Cost of a feature is mostly fictive if you don't have the metrics on how much that feature contributed to an end product - which is usually a small piece of the puzzle. Also new features are often built on your existing platform reusing years of previous investments which are hard to set a price for today. It will be much more expensive for the consuming party to build it without your platform, but exactly how much more expensive is fictive and experience-based estimates people frequently disagree on. Getting to a number for such metrics that can be defended is a challenge. Same for some of the other metrics like cost of a single incident. If you are lucky you could take it from history, like lost sales due to a certain issue or defect but more often this is not the case. Would love to learn how you found some of the $cost for your scenarios and if you feel you could defend them when put under pressure.

Hi @Bastijn Vissers great question, a few things to unpack here, I’m traveling right now but know that I saw the question and will reply later today!

Sorry, should not have said cost. I meant how you assign a dollar value to e.g. Productivity of an engineer or financial harm when the metrics you collect typically give quantities (#features delivered, #bugs solved, #incidents/time frame (reduced), etc). While the quantities are facts the dollar value for these quantities are hard to validate. You could of course translate back to cost of an SWE and relate to the amount of work an SWE does per time period. Multiply that by the number of consumers that use the feature(s) and state something on saving N-1 on engineering costs, or a bit less as platform components are more expensive to create and they still need to be integrated as well. It already becomes a bit fuzzy quick. Also it does not address the value of what was delivered, only saved costs. Just to be clear, I'm fully in favor of what you presented and do use it myself. I'm checking if maybe others -like you- have better ways of assigning value in ways that can be validated.

To be clear, those numbers are estimations, we are just trying to reduce uncertainty, we can't eliminate it completely. But in my experience a good estimation is normally all you need. So to answer your question. Productivity: I work with our product analysts on this numbers. When the company does planning, every team estimates the impact they will have next quarter in $. After the quarter is over they do a retro and calculate the actual impact they had. I use this retro number and extrapolate and ofc also divide by number of engineers. Stability: Again working closely with our analysts on this number, we know how much a customer contact costs us and we know from the past how many customer contacts are caused by Sev 1/2/3/4 incidents. We also know customers impacted by Sev and we know average revenue generated by a customer. For the validation of impact itself, I don't think you need to use $ numbers. E.g. we know one Sev 3 prevented per quarter has 10k£ impact. We now validate whether we prevented one Sev 3, but we don't need to validate whether there are additional 10k£ on the Wise balance sheet that we are responsible for.

Thanks. Stability would probably be similar in our company. Productivity I would need to think if/how our teams could actually estimate the impact they make next quarter in $. It is a few hops before our platform ends in a product in the market, assigning $ values at the internal platform dev team level is difficult. We did it of course but we more or less made it as low or as high as we desired (within reason) based on who we needed to convince. I did not find a way to base it on anything like previous/predicted sales, yet. Thanks for answering how you approach it!