“I’d love to adopt and champion metrics, but I’m afraid that if I do, I’ll lose my team. They’ll think big brother is always looking over their shoulder and start looking for jobs elsewhere.” This is a sentiment I’ve heard from a number of software development leaders when talking about trying to become more data-driven to improve their team’s productivity.
At this fall’s GitHub Universe, one of the best-attended sessions was on how metrics can help dev leaders. The presenter focused mainly on all the pitfalls of measuring the wrong things.
If we view data about our business this way, are we missing an opportunity to get better and actually improve our culture?
One of the keys to being successful with metrics is understanding what you hope to achieve before you start measuring. A smart leader once said “what you measure is your culture”. And, if that’s true, it could be reasoned that you’re leaving your culture to chance if you don’t have a clear set of metrics and KPIs being used across your entire team.
I worked for a company that valued customer success over top-line revenue growth. One way this manifested was that the sales staff was salaried without a commission. While each sales team had a goal of growing revenue, the individual sales reps did not have a quota. This meant that the reps were freed up to spend more time working with each client and they typically didn’t push products or services that weren’t needed just to hit a quota.
Leaders that want to use metrics to figure out who isn’t cutting it so you can let them go probably don’t have a great culture to begin with (and maybe need to re-evaluate their life choices in general). On the other side, leaders that focus on metrics that can help the entire team better achieve their goals can build a healthy and high performing team.
Some common healthy objectives to using metrics include:
- Increasing the speed at which you can deliver new value
- Improving the predictability and regularity of your delivery
- Enhancing the quality of your releases
- Balancing your teams investment into new stories with necessary non-functional work to ensure scalability, availability, and performance
- Curate a happy, innovative development team that rarely feels burned out
Metrics Do’s and Don’ts to Ensure Success
Ok, so we’ve acknowledged that done poorly, there can be pitfalls to using the wrong metrics. The right metrics, used correctly, can be transformational for modern software engineering teams. Here’s a few things I’ve seen work and not work that you may want to consider.
DO: Measure Metrics that Matter
The most important thing you can do when getting started is choosing the right set of metrics. We recommend metrics that make every contributor on your team more efficient. Efficiency drives the bus in several key areas, such as quality and delivery.
Metrics that EVERY software engineering team should measure:
Cycle Time: If you can only have 1 metric, make it cycle time. Cycle time measures the amount of time from work started to work delivered; in other words, the amount of time from first commit to production release. Cycle time is important because it is a proxy for how optimized your team is.
Culture Impact: Keeping an eye on cycle time has a direct impact on culture. Teams with shorter cycle times are more likely to hit feature deadlines predictably. In addition to operating more efficiently and feeling happier about hitting deadlines, this increases the trust your team has for each other and receives from other departments.
Tracking Your Cycle Time can uncover bottlenecks in your delivery pipeline
Investment Profile: Basically, your investment profile answers the question of “what are we working on?”. It measures how much time your team spends on different types of work, broken down into percentages. Your people’s time is your most precious and scarce resource. One of your primary jobs as a leader is to marshall those resources appropriately to develop a stable, feature-rich product.
Culture Impact: Managers that understand investment profile are able to be more empathetic. They are also better able to represent the interests and needs of their teams to others in the organization (i.e. “We can’t ship these 3 features AND issue a bug fix for customer x all this week. Let me show the data that explains why.”) Tracking investment profile is a great data-driven way to gain your team’s trust and loyalty.
Investment Profile tracks where your team spends their time
Quality Metrics: Software engineering teams are finding that quality equates to efficiency. Among the metrics that shed light on your quality, the best is your work breakdown. Work Breakdown measures the percentage of code creation containing new code, refactored code (change of old code), and reworked code (change of code that was recently released to production).
Culture Impact: No developer likes having their iteration disrupted to go urgently fix a bug found in production. It interrupts their ability to hit deadlines. Too many disruptions and you may have an issue retaining your best team members. Proactively managing rework today leads to happier teams tomorrow.
Your Work Breakdown shows how much of your team’s work is new, refactored, or rework.
DO: Pair Your Metrics with a Tactical Way to Improve
Tracking metrics that matter helps leaders understand performance and uncovers potential issues. But, as G.I. Joe said, knowing is half the battle. To drive impactful improvement, leaders need to understand how where and how they can make a positive impact. Tools that bring together metrics with work in progress deliver the visibility needed to drive proactive improvement.
For example, if your cycle time needs improvement, you might discover that you have a gap from when PRs are initiated to when they are reviewed and merged. This is a common scenario. Long living PRs represent a number of issues for a development team. They slow value delivery, cause quality issues, and have a negative effect on internal processes and culture. Pay attention to your PRs and make sure they don’t languish in limbo to unclog your delivery pipeline.
Analyzing pull requests gives software leaders a good idea of where they can start improving
Culture Impact: This is super important for cultivating a culture of improvement. Leaders that focus just on the numbers and aren’t able to provide a tangible path to improvement for their team can run the risk of damaging culture. But leaders that can identify issues and provide an improvement path, will help their teams grow the skill level and have happier teams.
DON’T: Ignore the implications of what you measure
A common concern of implementing metrics is that the system will be gamed; that developers will work towards metrics success rather than actual team success. In the early days of measuring engineering productivity, some tried to use the number of lines of code as a metric. Makes sense on its face, more productive developers write more code, right? The end result of this for those looking to hit their metrics is that an edit that could be written in 10 lines of code was written in 50. There’s no incentive to look for the most efficient streamlined solution. In fact, as a developer, you would be incentivized to find the bulkiest solution. This leads to poor performance, not exactly what was intended.
Culture impact: Choosing to focus on the wrong metrics can create a culture where your team is working towards a number rather than towards a goal. Not only does this mean that people will game the system, but also once they hit their number for the week, they will be less engaged. The reality is that some thought about what to measure and how to communicate with your team will go a long way.
DON’T: Use Story Points to Measure Success
Never, ever. If you aren’t already sure why this is a terrible idea, Capgemini explained it in great detail at last year’s QCon. Check out the presentation: https://www.infoq.com/presentations/metrics-evaluation/
DON’T: Use the Same Metrics to Measure Every Contributor
This may sound counter-intuitive, but hear me out. Any manager, coach, leader, parent, etc knows that everyone on the team operates differently. Different strengths and weaknesses. Different ways they feel appreciated or are incentivized to improve.
When rolling out metrics to the team, work together with each team member (or team lead) to understand what they think is most important or areas they want to improve, and then help them understand the metrics that you can both use together to help them achieve their goals.
For example, if you have a developer that wants to become a team leader, then show them their review depth. If you have a team leader that is worried about their iteration being interrupted by production bugs, help them understand investment profile and work breakdown.
Culture Impact: At the end of the day, developers understand that they need to deliver value to the organization and that at some level they will be measured on their performance. But they also have personal improvement goals that are important to them. Working with them to agree on a set of metrics keeps everyone on the same page and makes them feel more included in the process.
DO: Use Metrics to Find Teams and Contributors that are Stretched too Thin
A great way to use metrics is to help your team scale intelligently. Use team-based metrics to discover which teams are maxed out and need help to hit delivery deadlines.
Culture Impact: This is a great way to improve culture using metrics. If you can add a new resource to your team or better balance workload, suddenly your team will love metrics (and you) because it has a direct impact on their daily work life.
DON’T: Use Metrics Alone to do Performance Reviews or Determine Raises
Maybe the worst use of metrics is directly tying compensation to metric performance alone. (Imagine compensating based on lines of code written :O). Metrics can be a part of a healthy performance review/one on one, but they shouldn’t make up the entire picture. I recently met with a LinearB customer that told me “I use the metrics in LinearB to validate and check my gut feeling and anecdotes about performance.” This is a much better use of metrics.
Culture Impact: As a leader, you probably already have a good idea of who the top performers are on your team. Use metrics to help you validate that feeling or show you where you might be off, then have a conversation with your team about what you find. Your team will appreciate the opportunity to have a productive conversation that includes data in addition to feeling like they are working for someone that cares about them as a human.
Start Here
If you’ve read this far (thanks :)) you may find yourself caught between wanting to be more data-driven and concerns about ruining what you think is a pretty good culture. Hopefully, we’ve shown you that using metrics can help you improve your delivery AND your culture. But where is the best place to get started?
The answer can be found in team-based metrics. Cycle time is a good example. If you can improve your team’s cycle time, it is a rising tide that lifts all boats. The same can be said for the quality metrics like rework %. These two key metrics can give you some quick wins and help your team understand how you plan to use metrics. It also helps you diagnose issues that impact the whole team and can improve your processes.
Changing a culture is hard. If you’re on a team that’s scaling or about to scale, now is the perfect time to install a data-driven culture. Getting started now will be easier than changing things up later.
You can use metrics without damaging your culture. It turns out that it’s not the insane tightrope walk many make it out to be. Choose the right metrics, incentivize the behavior that helps your entire team, and be empathetic when you share metrics with your team.
