Software engineering metrics are crucial. By using metrics, teams and organizations can measure and improve operational efficiency and business outcomes in clear and objective ways. In addition to empirically measuring and benchmarking engineering KPIs, the insight a holistic metrics program provides leads to better decision-making; resulting in more team productivity, better quality, and higher customer satisfaction.
But how do you choose the right metrics to focus on? There are lots of them, and they aren’t equally valuable–looking at you Velocity and Lines of Code. This is very true of one of the most popular tools in the R&D ecosystem: Jira. It’s chalked full of metrics and data, but not necessarily utility. As an example, let’s look at story points. It’s probably the most common metric in Jira, but not the most helpful at face value. Why? Because it’s easily gamed: One team’s 10 is another team’s 3.
In this blog, we’re going to dig into:
- Why metrics are so crucial in software development
- The type of metrics that are commonly used by engineering teams
- The five most valuable Jira metrics your team should consider adopting.
Let’s get to it.
The Importance of Metrics for Engineering Team Operations
Despite software engineering being a STEM discipline, it’s riddled with unscientific reasoning for why things are done a certain way–including some common logical fallacies like cargo cult and appeals to authority. The net result is nothing improving and the continuation of inefficient operations because “that’s the way it’s always been done.”
But when a team builds a holistic metrics program–one that includes visibility, industry benchmarks, and clear improvement goals–they’re well on their way to operational excellence and business impact.
So the question then becomes: “What metrics should I measure and set goals against to improve my engineering team?” The answer is a blended approach that includes both leading and lagging indicators, operational and business metrics, and KPIs that can easily translate into improvement OKRs at the team and org level.
What Are the Different Types of Metrics?
There are many software engineering metrics out there. Also, there are different ways in which you can categorize the different metrics available. For instance, software engineering metrics can be grouped into source code metrics, development metrics, and testing metrics.
- Source code metrics are measurements you can obtain from the code itself, such as PR size.
- Testing metrics refer to measurements related to automated tests, especially unit tests. Branch coverage is a classic example of a testing metric.
- Development metrics refer to the development process itself as work progresses from the coding stage to release into production. Cycle Time is a great example.
When it comes to Agile metrics specifically, a popular categorization consists of dividing them into Lean, Kanban, and Scrum metrics.
What Are the Metrics Used in Agile?
As you’ve just seen, there are not only a large number of software engineering metrics but also several ways to categorize them. So, how can you know the metrics that Agile teams really use?
Keep in mind that the term “Agile” usually refers to a large number of similar yet unique development methodologies. The actual metrics teams use rely heavily on the “flavor” of Agile development they favor, such as Scrum or Kanban. Here are some examples:
- A Scrum team will likely find value in planning and execution metrics known as Accuracy Scores. These two metrics tell a story about a team’s ability to scope work, prioritize planned tasks, and adjust to the ebb and flow of engineering.
- A Kanban team will likely find more value in Flow and Throughput metrics as there isn’t a time-boxed iteration to plan and measure execution against. Instead working to maintain Flow consistency week-over-week and increasing gradually over time.
Scrum is what most Agile teams and organizations favor, so let’s explore the metrics in Jira that matter most to development teams.
The 5 Jira Metrics That Matter Most for Your Dev Team
Note: Jira is an amazing tool that is invaluable for development teams. That said, getting the most value out of it requires attention to data hygiene and ensuring that Jira best practices are followed.
1. Velocity
Velocity is arguably the most well-known metric, not only in Jira but for Agile in general. Velocity, as its name suggests, is about how fast your team is going. The value of your velocity metric refers to how many story points the team completes over time.
Velocity is an effective metric when used in the context of a single team. By continuously measuring velocity, you get better at setting sprint goals and sprint capacity planning. Unfortunately, many software teams routinely abuse this metric and suffer the consequences of doing so.
The most common mistakes are using velocity for comparison between teams and as a performance indicator, which can damage team morale and (incorrectly) motivate developers to game the metric. The key takeaway here is that velocity can be an effective metric, but only when used correctly.
2. Cycle Time
Cycle Time is the amount of time the team spends working on issues, from the start of work until the feature reaches the end-users.
Cycle time is an incredibly valuable metric because it can help you discover inefficiencies in the whole software development process. Here’s a fairly common scenario:
- A large pull request (>250 lines of code changes) is issued
- It takes longer to be picked up and reviewed because it requires dedicated time to do so
- It sits idle and the issuer and reviewer move on to other priorities
- The cognitive load for both parties increases as time goes on as they have to reorient themselves to the work
- This all has a compounding effect on total cycle time, which is steadily increasing.
- This all drives efficiency and quality down, leading to higher rates of rework and code churn.
Our data science team studied the cycle time and other KPIs of over 2,000 teams and discovered the average cycle time for the developers studied was 6 days + 5 hours.
Want to learn more about being an elite engineering team? Check out this blog detailing our engineering benchmarks study and methodology.
A great way to reduce the burden of the PR review process, increase quality, and lower cycle time is to automate the process using programmable workflows.
3. Defect Ratio
If you want your Scrum project to be of the highest quality possible—and who doesn’t?—you must be aware of how many defects are being introduced into the project, how long it takes (on average) for them to be fixed (known as Mean Time to Restore), etc. Using Jira capabilities, such as filters and dashboards, you can create sprint reports to depict the defect ratio of a given sprint, epic, or version.
4. Sprint Burndown
The sprint burndown chart shows the amount of work the team members still have to complete in the current iteration. Ideally, the team should be able to finish all the work–or come very close–it is committed to finishing during the sprint.
By keeping track of the sprint burndown chart, along with other Jira sprint metrics, can help you learn important things about team’s health and productivity. You might find out that the team is committing to more work than it can handle–or not taking on enough work.
Pro Tip: Be sure your teams are taking on the right amount of work. Always hitting 100% of your sprint commitments may indicate that a team needs to increase their scope of work.
In short, keeping track of the sprint burndown chart and evaluating how close the team got to delivering the work that was expected can help you diagnose and fix anti-patterns in your organization.
5. Epic Burndown
In Agile jargon, “epic” is the name you give to a big user story. You then break it into smaller user stories that can fit into a single script. It’s essential to be able to evaluate how much work the team has done on each specific epic, and how much work is still left.
By viewing the epic burndown chart, you can see the progress your team is making toward the completion of a bigger project. You can then extrapolate their speed over a longer timeline to estimate how many more sprints they’ll need to complete the epic, which assists in your planning.
Jira Metrics Might Be Effective, But They’re Not Enough
You can’t improve what you don’t measure. That’s true for many industries, and software engineering is no exception. And there are a plethora of software development metrics to measure.
Unfortunately, that abundance has its downside: it can be overwhelming to evaluate the available metrics and decide which ones are worth keeping track of. This post was about some of the main Jira metrics that can be helpful for dev teams.
However, it’s important to keep in mind that Jira metrics aren’t a panacea when it comes to measuring and working to improve your team’s performance. As you’ve seen, managers and even developers often misuse some metrics. That, in turn, might encourage developers to game the metrics, hurting team culture in the process.
Also, data from project management software like Jira can be inaccurate, and the picture you get from it might be lying to you.
Do you know what doesn’t lie? Your commits. Combining project data with statistics from your repository is a great way to maximize the benefits of your metrics–which is the inspiration for DevOps Research and Assessment (DORA) metrics. These four metrics are the closest the software development community has come to quantifying productivity.
LinearB offers a totally free forever DORA dashboard that provides both project and git metrics in a single view. Not only that, it also includes in-app benchmarks as well as leading indicator metrics–like PR size and Merge Frequency–that teams can use to improve DORA metrics.
