Why computing DORA metrics is hard

The DORA metrics are a set of metrics that measure the quality and velocity of software delivery of an engineering organization. By measuring and continuously iterating on these metrics, engineering teams can deliver better software to their customers faster, and achieve significantly better business outcomes.

So why aren’t those metrics widely available in your engineering organization? Because they are difficult to compute. Here are a few of the hurdles we faced when we were building Salesforce Einstein, and the reason we started faros.ai. Keep these (and us!) in mind when you start this journey.

Let’s start with the time to restore service metric. Good news: you can probably get this metric from your incident management system. Assuming you are OK with not having a single pane of glass for your metrics. But don’t get ahead of yourself!

Now let’s look at change failure rate. This is a metric that combines your incident and your deployment data. This means you need to extract and load data from both sources somewhere to compute the metric. You cannot just get it from one of your systems. That’s just the first difficulty. Next you need to correlate your incidents with your deployments.

Then, let’s take a look at weekly deployments. It is common for Engineering organizations to have several CI/CD systems. The data from the different systems will need to be modeled and normalized. More fun!

Lead time for changes ups the ante: not only do you need data from multiple systems (commits, pull requests, build, artifacts, deployments), that data needs to link properly. Your system will need to be robust enough to gracefully deal with missing data and out-of-order ingestion. Most likely you will also need to impute changesets for your deployments.

Finally, if you overcome those difficulties, you will want to roll up those metrics along organization lines, slice and dice by application, by stage, and over time. All of a sudden, you have to maintain a service catalog, deal with hierarchical roll ups and drill downs, …

TLDR: DORA metrics are difficult to compute; don’t build them from scratch. Use tools and products that deal with these difficulties to get to value faster. Check out Faros Community Edition.