Setting a target on a metric won’t change people’s behavior, they will only learn to game it
There are various metrics currently used to measure the development team’s performance, process, the health of the code base, or the health of the product overall. The problem is that there is no single metric that matters, regardless of what you hear about any of the available metrics, all of them are good and bad at the same time.
The thing is: any given metric, used in isolation, tells only half of the truth, at best.
Yet, it is very common, too common, to set a target on a given metric.
The problem is not a young one, there is even a law called Goodhart’s law that says:
"When a measure becomes a target, it ceases to be a good measure."
The issue boils down to this:
if you set a target on any given measure, people will eventually game this metric to achieve the target. It will not be the leap in performance that happened, it will simply be a change in how people report performance.
Yes, this also applies to KPIs.
Treat the metrics as the product’s EKG results
There are many “articles” claiming that this or that metric is bad.
Velocity: bad.
Cycle time: bad…
The problem is that all of them are good when you know how to use them.
Treat your metrics like a doctor would treat an EKG or lab test results. If they find any potential indication of an underlying problem in your results, they will most likely try to confirm it with other tests, and if the problem is confirmed, they will propose a solution. However, it wouldn’t be of any use for a patient if they were told: “lower your blood pressure.” What they would want to be told is what changes they should introduce into their lifestyle to achieve the desired outcome.
That’s the way you should look at metrics as well.
Start using metrics to your advantage by applying the 4 golden rules
1. Never trust a single metric to give you the full picture
Using one metric can be deceiving because in order to understand what one metric tells us, we will potentially need another metric.
An example of this may be a Velocity of a Scrum team measured by a summary of Story Points delivered per sprint alone. What if the amount of points is stable, yet the number of stories/tasks is dropping with every sprint? What if the cycle time or a lead time is rising simultaneously? Is everything still all right?
2. Keep metrics objective by not setting targets on them
Instead, observe them, interpret them, and compare them against each other.
If you find an undesirable trend or a pattern, implement a change in your process, then observe if anything improves.
3. Not everything is worth being measured
One of the metrics, not worth tracking are lines of code written per day. Although this metric may seem like a straightforward measure of productivity, it fails to take into account the quality and efficiency of the code and can even incentivize developers to write unnecessary or redundant code to meet an arbitrary quota.
4. There’s no one-size-fits-all when it comes to metrics
Different organizations and different products will need different metrics. The best way to find your perfect fit is through experimentation.
Here are a couple of suggestions of metrics that often prove handy in analyzing a team’s performance:
Velocity and/or cycle/lead time + throughput for changes — this combo will help you better understand how many changes your team processes in a given time period.
Change failure rate + bugs per period/cycle time — these metrics will allow you to better understand how the quality of the delivered code changes throughout the project’s lifetime.
Pull requests cycle time and/or pull requests cycle time per service/part of the repository + pull requests per developer per iteration + code review cycle time — this set will give you a better understanding of how ease of introducing new features or technical debt changes throughout the project.
Get to know your metrics before introducing them to the project
The better you know the metric, the more insight you will draw from its analysis, and the more precisely you will be able to identify areas for improvement. Learning the ins and outs of the metrics will also allow you to avoid common mistakes that could distort the data and lead to the wrong conclusions being made.
Top comments (0)