Objective data to measure software development is here, and it’s here to stay.

For a long time, the notion of using such data was thought to not really be possible. Thought leaders like Martin Fowler and Joel Spolsky basically said it couldn’t be done. Clearly, it’s a challenging task that frustrates software development managers everywhere. Shoot, I wrote an article way back when basically arguing that it is impossible to do.

Well, I’d continue to argue that it was impossible to do. But now, with the rise of tools like Git, Jira, and other project management tools, it started becoming clear that the data is there to enable us to get a closer, more data-driven look at what is going on inside software development projects. That data just had to be revealed.

Table of Contents

What Are DORA Metrics?

One of the most important and well-known studies was done by the DevOps Research and Assessment (DORA) organization. They surveyed thousands of DevOps engineers and leaders over six years, coming up with a set of four key metrics that were considered critical to the success of software development projects.

The four DORA metrics are:

  1. Deployment Frequency
  2. Mean Lead Time for Changes
  3. Mean Time to Recovery (MTTR)
  4. Change Failure Rate

The first two metrics, deployment frequency and mean lead time for changes, measure the velocity of a team. MTTR and change failure rate measure the quality and stability of a project. All four metrics can be derived from mining the tools that you are currently using. 

These four DORA engineering metrics are designed to allow software developers to align their work against the goals of the business. They have become the standard way for CTOs and VPs of Engineering to get a high-level overview of how their organizations are performing.

By keeping an eye on the DORA metrics and organizing their work around improving them, development teams can ensure that they are doing the right things to move their projects, and more importantly their business, forward.

Of course, understanding what the metrics actually measured and what they mean is necessary to make them useful. In addition, knowing the current state of these metrics is required for improving them as you move forward.

So let’s take a look at these four key measures.

Deployment Frequency

What Is It?

Deployment frequency measures the number of times that code is deployed into production. It’s usually reported in deployments per day or week.

Deployment frequency with engineering benchmarks from LinearB

Now, production can mean different things to different customers. For a SaaS company, it normally means actually delivering code to the production platform used by actual customers. For other companies, it might mean “made a version available for use by customers.” 

Why It’s Important

Increasing deployment frequency is an indication of team efficiency and confidence in their process. A team that can deploy more frequently is moving work through their pipeline faster and being more efficient about all of their work products.

How Is It Calculated?

It tallies the total number of deployments an organization does in a single day. As noted, the definition of “deployment” can vary between organizations. This metric can be automated if a team has a Continuous Integration/Continuous Delivery(CI/CD) tool that provides an API into its activity.

How Do You Improve It?

If you wish to improve your deployment frequency, you should invest in:

  • Improving automated test coverage
  • Integrating with CI/CD tools
  • Automating the release validation phase and release process
  • Reducing the error recovery time on production

Mean Lead Time for Changes 

What Is It?

Mean lead time for changes is the average time it takes from code being committed to that code being released into production.  

Some organizations begin tracking the time from the first commit of the project’s code, while others measure it beginning from merging the code to the main branch.

Many organizations roll mean lead time for changes into a metric called cycle time, which is discussed below.

Why It’s Important

A lower mean lead time for changes means that your team is efficient in coding and deploying projects and is adding value to your product in a timely manner. Attempting to lower the average incentivizes teams to properly divide the work, thoroughly review the code, and have a fast deployment.

How Is It Calculated?

Each project is measured from start to finish, and an average of those times is calculated.

How Do You Improve It?

This metric can be improved by:

  • Break projects into smaller and more manageable chunks
  • Create an efficient code review process
  • Adding automation to the deployment process
  • Ensuring that the CI/CD process is as efficient as possible

Going Above and Beyond with Cycle Time

LinearB goes beyond the DORA metric of mean lead time for changes to provide cycle time.  

cycle time benchmarks

Cycle time is a powerful metric that measures how long it takes a given unit of code to progress from branch creation to deployment in production. It’s really a measure of how fast a given task or subtask gets delivered to end-users. And of course, actually delivering functionality is the purpose of every development organization.

Cycle time is divided into four subsections:

  1. Coding Time – Normally measured as the time between the first commit to a given branch and the moment a pull request is created for that branch.
  2. Pull Request Pickup Time – This is the time to respond to a pull request being created, i.e. when the review for that pull request starts.
  3. Pull Request Review Time – The time between the pull request review starting and the code being merged.
  4. Deploy Time – Deploy time is the span between the merging of the code and that code being deployed to production.

Improving and measuring cycle time has a number of benefits:

  • Closely tracking coding time encourages you to divide work into smaller, more manageable chunks. Cycle time goes up if a given branch or project is big and takes a long time. Instead, it encourages smaller bites of work.
  • It pushes the team to process pull requests in a timely manner. It helps prevent languishing pull requests and pull requests that are too large to review effectively.
  • Teams that track deployment time are motivated to focus on improving and streamlining build and deployment processes.

Rising cycle times can be an early warning system for project difficulties. If I had to pick one thing for a team to measure, it would be cycle time.

Measure Your Team's Cycle Time in Minutes. Break down your team’s cycle time in 4 phases for bottleneck detection in seconds. 
Get context in one click and dive into any of your cycle time metrics to discover which branch is behind the spike. 
Diving into your data has never been this easy!

Mean Time to Recovery (MTTR)

What Is It?

Mean time to recovery, also known as mean time to restore, measures the average amount of time it takes the team to recover from a failure in the system. 

“Failure” can mean anything from a bug in production to the production system going down.

mean time to restore

Why It’s Important

Obviously, downtime is not good, and the quicker a team can recover from it, the better. 

Keeping an eye on mean time to recovery will encourage the building of more robust systems and increased monitoring of those systems.

Quick recovery and response times are a reflection of the team’s ability to diagnose problems and correct them. Measuring mean time to recover can have the effect of making the team more careful and concerned about quality throughout the entire development process.

How Is It Calculated?

Normally, this metric is tracked by measuring the average time to resolve the failure, i.e. between a production bug report being created in your system and that bug report being resolved. Alternatively, it can be calculated by measuring the time between the report being created and the fix being deployed to production.

How Do You Improve It?

MTTR can be made better by:

  • Building a CI/CD system that quickly reports failure
  • Ensure there is a process in place to take immediate action on failures
  • Prioritize recovery from failure over all other tasks
  • Improve deployment time

Change Failure Rate

What Is It?

Change failure rate measures how often a code change results in a failure in production. Changes that result in a rollback, in production failing, or in production having a bug all contribute to this metric. 

change failure rate

Why It’s Important

This metric is important because all time spent dealing with failures is time not spent delivering new features and value to customers. Obviously, lowering the number of problems in your software is desirable.

How Is It Calculated?

Normally, this metric is calculated by counting the number of times a deployment results in a failure and dividing by the number of total deployments to get an average number. A lower average is better.

How Do You Improve It?

Change failure rate is improved when you:

  • Ensure all new code is covered by automated unit tests
  • Improve automated testing as part of your continuous integration process
  • Do thorough and complete code reviews to help prevent issues being introduced into production

The Benefits of Tracking DORA Metrics

Decision Making

Consistently tracking DORA metrics will enable you to make better decisions about where and how to improve your development process. Doing so will reveal bottlenecks, and enable you to focus attention on those places where the process may be stalled. Trends can be identified and the quality of decisions about what was focused on can be validated. 

DORA metrics tracking can help focus both the development team and management on the things that will really drive value. They allow you to make decisions based on data rather than merely a finger in the wind or a gut feeling.

Delivering Value

DORA measures the value that your team is delivering. If your DORA metrics are favorable, your team is delivering value to your customers and maintaining the quality necessary not to be distracted from that focus. And that’s pretty much the bottom line for any business, delivering value to your customers.

Virtuous Cycle

When anything gets measured, it will likely be gamed. That is, people will change behavior to optimize that which is measured. Many times this can have a negative, distorting effect on what a development team does.

DORA metrics can be gamed, but the great thing is that you want them to be gamed. You want your team working to optimize these metrics. Gaming them results in good things.

Normally, gaming a metric has a negative impact on teams, but these metrics were carefully devised to do the exact opposite, create high-performing teams. Since they highlight inefficiencies and wasted time, gaming them will increase efficiency and reduce waste. 

How to Measure and Improve DORA Metrics 

DORA metrics are important, and LinearB allows them to be tracked easily. We give you a DORA metrics dashboard right out of the box that can be easily displayed and tracked.

DORA Metrics Dashboard

A dashboard like this could be useful by giving senior members of your software development organization a higher-level view of the DORA metrics for the organization. With this simple view, leaders can see at a glance how the team is doing and what mid-course corrections might need to be made.

In addition to the actual DORA metrics themselves, LinearB can track other metrics that can help improve your organization’s performance. 

Leading indicators like pull request size, pull request review depth, and pull request review time can all be monitored and when improved, will reduce mean lead time for changes and deployment frequency.

WorkerB Improves DORA Metrics

Idle time is the time spent waiting for things to happen in your software development process. Pull requests sitting idle and unreviewed is a good example. It is a big killer of two important DORA metrics: deployment frequency and mean lead time for changes.

WorkerB is a developer bot that can have a drastic, positive effect on reducing idle time and thus improving your DORA metrics. For example, if the team wants to reduce cycle time, a leading indicator the team can set a goal around is reducing PR size.

WorkerB will send out an alert on Slack or MS Teams if a PR has been created that exceeds the goal. WorkerB even reduces context switches by enabling developers to create Jira issues for unlinked branches and even review PRs smaller than 5 lines from right within Slack to help devs stay in a flow state.

LinearB customers routinely report more than a 50% reduction in cycle time in their first four months of using WorkerB.

Discover Your DORA Metrics in Minutes

DORA metrics are based on years of research into what really matters for software development teams. Focusing on them will result in more value being delivered through your development pipeline. More value means happier customers. 

LinearB can help your team track them consistently and thus bring about a profound and lasting impact on your software development process and your business. Schedule a demo today!

datadriven engineering starts with DORA metrics
LinearB provides DORA metrics right out of the box so you can discover your DORA metrics in minutes. Get started today!