Hey, folks!
The State of DevOps lists 2 metrics for Software Development: Deploy Frequency (Throughput), and Lead Time for Change. They are great for measuring the expected outcome of an Engineering Team.
I've seen managers attaching it to OKRs or used as the area KPIs. They are excellent indicators of performance for companies.
However, as an Engineering Manager, I don't see they're so useful for daily activities. They provide a high-level overview, but little visibility on what's going on in daily activities.
Managers usually need a more deep analysis of the process to improve the Lead Time and Deploy Frequency themselves.
For that task, I would go break Lead Time into smaller steps of the work, such as the Time from Commit to Open a Pull Request (assuming the company is doing code review), Time to Merge (from Opening to merging a Pull Request), and so on.
How do you track process healthiness and effectiveness in your company?
Top comments (2)
Your most basic metric of process health is points delivered per sprint per team. There are dragons here and it takes a lot of nuance to get this right. Never have a conversation about this metric with an engineer because discussing points breaks points. See Goodhart's Law
The best way to use it is just to track it over time. When there is a change to the trend have a conversation with the team lead. The topic of the conversation is there anything interesting that happened this sprint? Gather more data. See if there is anything to learn. Changes in points both directions are a hint to you to dig deeper and see how everyone is doing.
Another good metric is time from ticket first goes to "in progress" to "complete". Tickets that hang around for a long time in a in progress state are a indication that size of your tickets may not be quite right.
I can go on and on about the kinds of metrics you can track about teams and engineers. But I cannot stress this enough. There is no magic bullet number for process health. Every data point must be treated as directional or you risk creating a really toxic engineering culture.
Hey, Stephen
Thanks for the answer! I stopped estimating User Stories in Story Points because I've seen the team having time-consuming discussions on whether a given US worth 1 or 2 points.
So, instead of estimating, I like to have some discussions on "how to implement given User Story." It's a good practice that aligns the team and turns anyone eligible for developing it later. In other words, it avoids queues and improves the overall speed of development.
So, instead of measuring Story Points, I rather measure the number of delivered User Stories.
When it comes to the metric from when the ticket first goes to "in progress" to "complete", I'm total with you!
I call this metric Lead Time, however, some people call it Cycle Time. I explained why I don't like Cycle Time in this article.
I also wrote another blog post called 50 Shades of Lead Time that explain how to apply this idea to other steps of software development.