A high-performing software engineering team is integral for businesses to deliver value. Understanding and improving engineering performance starts with a well-structured engineering metrics program. With the insights provided by implementing an engineering metrics program, you can deliver high-quality work efficiently and align your work with business goals.

Understanding Engineering Metrics

Engineering metrics are quantitative measures of engineering team performance and health. When used correctly, you can leverage engineering metrics to find improvement opportunities, track progress, and demonstrate value to stakeholders. The key metrics to focus on are lagging indicators, which reflect past performance, and leading indicators, which predict future performance.

Lagging Indicators:

Lagging indicators measure the outcomes of past engineering performance and provide insights into the effectiveness of your processes over time. These metrics are essential for understanding the long-term health and stability of your engineering practices.

  • Cycle Time: The time from the first commit to production release, and the related four phases: coding time, pickup time, review time, and deploy time.
  • Deployment Frequency: How often you deploy code to production over time.
  • Mean Time to Restore (MTTR): The average time it takes to restore from a failure of the system or one of its components
  • Change Failure Rate: The percentage of production deployments that cause one or more failures.

Engineering metrics dashboard showing the DORA Metrics segmented by efficiency (cycle time and deploy frequency) and quality metrics (CFR and MTTR)

Leading Indicators:

Leading indicators predict future performance and help identify potential issues before they escalate. Monitoring these metrics allows teams to proactively address inefficiencies and improve their processes.

  • Pull Request (PR) Size: The number of code lines modified in a pull request. 
  • PR Pickup Time: The time between when someone creates a PR and when a code review has started.
  • PR Review Time: The time between the start of a code review and when someone merges the code.
  • Rework Rate: The percent of total code that has been reworked over a specified period.
  • PRs Merged Without Review: The number of code changes that enter production without peer review.

Engineering metrics dashboard showing goals for PR size, pickup time, review time, PR lifecycle, merged without review or merged with basic review

Setting Up Your Engineering Metrics Program

1. Visibility: The Foundation of an Engineering Metrics Programs

The first step in building your engineering metrics program is achieving visibility into your software development lifecycle (SDLC). Using a Software Engineering Intelligence (SEI) Platform like LinearB, you can aggregate data from various sources such as project management tools, git repositories, and CI/CD pipelines to create a comprehensive view of your engineering processes and outcomes. 

Integrations graphic

Once you aggregate data your data from all relevant tools and platforms, you can start to correlate data points across the different sources to achieve a holistic view. Analyzing historical data over time will unlock insights into the trends and patterns of your engineering organization’s performance.

2. Benchmarking: Understanding Your Position

Once you have visibility, the next step is benchmarking your engineering metrics against industry standards so you can figure out what they mean. 

  • Is your 12 day cycle time good? 
  • How about your average PR size of 250 lines of code? 

Visibility is a great first step, but understanding the context, where you are, and how far you have to go is much more important. All the engineering metrics and visibility in the world don’t mean much without a baseline. The only way to know is to contextualize your metrics and see what your industry peers are doing using the Engineering Benchmarks.

EngineeringBenchmarks-Page5.png

LinearB analyzed the engineering metrics from more than 3,600,000+ pull requests (PRs) from over 2022 organizations to illustrate what the top performing teams’ metrics are in each of these key areas. This helps you understand where your team stands and identify areas that need improvement.

3. Diagnosis and Reporting: Identifying and Addressing Bottlenecks

After benchmarking, shift your focus to diagnosing issues and reporting on improvement opportunities. Use metrics to pinpoint bottlenecks in your processes and develop strategies to unblock them. We recommend most organizations start with cycle time, because it provides an end-to-end snapshot of software delivery efficiency, and is the source of the single most common developer productivity bottleneck: code reviews.

Diagnosing Common Cycle Time Bottlenecks:

  • High PR Sizes: PR size is a leading indicator for cycle time and software quality. Smaller, more manageable pull requests help developers minimize the risk of introducing new bugs or causing production issues. When PRs are concise, they are easier to test and review, allowing developers to focus on specific functionalities without being overwhelmed by numerous changes.
WorkerB notification displaying an alert when a PR was opened with too many code changes.
  • Slow PR Pickups: The longer code waits for review, the bigger the context switch for all involved participants. Long pickup times and excessive code review notifications reduce developer productivity and negatively impact developer experience. You should optimize review routing and empower engineering teams to better manage the code review process.
WorkerB notification displaying an alert when has been waiting for too long.
  • Excessive Rework: Code rework is a leading indicator of team frustration because it is often a symptom of a misalignment between product and development. This misalignment could result from process failures, incomplete requirements, a lack of skills, poor communication, excessive tech debt, poor testing practices, architectural problems, or high code complexity.
WorkerB notification alerting when there's high risk work.

Building an Effective Improvement Strategy

After gaining visibility, leveraging benchmarks, and pinpointing bottlenecks, the next step is developing an improvement strategy. This involves setting specific, measurable engineering goals and using automation to maintain focus and consistency.

Steps to Build Your Improvement Strategy:

  1. Set specific goals – focus on one or two key engineering metrics per quarter and use benchmarks to set realistic, achievable targets. An example goal could be to reduce PR review time by 20% within the next quarter.
Engineering metrics goals for efficiency and quality.
  1. Create custom dashboards to track your progress. These dashboards should include both leading and lagging indicators to provide a comprehensive view.

Engineering metrics dashboard showing cycle time, coding time, pickup time, review time, and deploy time.

  1. Automate processes wherever possible. Tools like LinearB’s Software Engineering Intelligence Platform can help automate routine tasks and enforce PR policies. Set up automated alerts for key metrics to keep your team focused on improvement goals. For example, you could use automatic notifications for PRs that exceed size limits or have pending reviews.
Use a goals dashboard and automated notifications to keep your team on track.

Pinpointing Bottlenecks with Data Segmentation

When building your improvement strategy, you need to segment your data to address the right issues with precision and efficiency.

Common Data Segments to Analyze:

  • Team-Based Bottlenecks: Identify issues that are specific to certain teams.
  • Repository or Code Base Segments: Focus on certain repositories or parts of your codebase that may be problematic.
  • Service-Specific Analysis: Analyze services that involve multiple branches and repositories.
  • Custom Metrics: Use labels and tags to create custom metrics for targeted analysis.

Engineering metrics dashboard showcasing how different segmentations affect cycle time.

Pro Tip: Start with a dashboard that includes DORA metrics and leading indicators like PR size and pickup time. This can act as your foundation for recurring team syncs and help identify where improvements are needed.

Dashboard displaying DORA metrics you can use to showcase engineering health.

Once your dashboards and different segmentations are in place, you can start digging into the metrics and find the root causes of operational issues. Granular analysis will help you uncover process bottlenecks and inefficiencies.

Common Bottlenecks to Investigate

This is by no means an exhaustive list, but our customers–and even our internal teams–have mentioned that these are points of friction that they encounter with some frequency. Take a look and see if any of these look familiar. Remember to compare these metrics to Engineering Benchmarks to gain more context and define “good” for your team.

Stuck Issues 

Identify tasks or issues that are not progressing due to operational bottlenecks, lack of resources, or because something got lost over the normal course of business.

Cycle Time - Pickup Time Emphasized

Engineering Metrics to Investigate: 

Overloaded Engineers 

Prevent burnout and inefficiency by ensuring developers are taking on the right amount of work, have balanced workloads, and take time to rest and recharge.

2022-03-29-13-48-51 (1).gif

Engineering Metrics to Investigate:  

  • People Activity (WIP, Active Days, PRs Opened, Reviews, Active Branches)
  • Resource Allocation
  • Investment Strategy 
  • Team level Project Forecasting Insights (git activity, resource management and over extended developers)
  • Accuracy Scores (is unplanned work bogging the team down)
  • Custom Metrics (Like Bugs in Production)
  • Developer Coaching Dashboard** (make sure developers have a healthy balance of engaging problems to solve and work to keep the lights on–like bugs and addressing tech debt)

**Coming Soon 

Overlooked Reviews 

Whether through the natural back and forth of the PR process, lack of visibility, heavy cognitive loads, or lots of context switching, sometimes work falls through the cracks. This common issue can impact quality and slow down code reviews–not to mention the entire development pipeline–because both authors and reviewers have to reorient themselves with the work to be done. This often results in compounded delays, lots of frustration, and shallow reviews (“LGTM” on 300 diffs isn’t a review). 

PR Pick Up Time + Goals

Engineering Metrics to Investigate: 

Conclusion

Building your engineering metrics program is how you can drive continuous improvement and achieve business objectives. By setting clear goals, benchmarking against industry standards, and using effective reporting, engineering leaders can:

Leveraging Software Engineering Intelligence Platforms like LinearB will streamline these processes, giving you the visibility and automation needed to succeed and fulfill the dual mandate of operational excellence and business impact.

By focusing on visibility, benchmarking, and targeted improvements, your engineering team can not only meet but exceed its performance goals, driving significant value for your business.

Want to learn more about building an engineering metrics program? Download the Engineering Leader’s Building a Metrics Program

Eng Leaders Guide Metrics Program - 1280 x628 V1.png