There are two BIG questions that you’ve likely been asked repeatedly by leaders (like your CEO) over the years that are intrinsically linked to business priorities:
- When will feature ABC be ready?
- Can you deliver it sooner than XYZ?
Historically, answering these questions has been an exercise in divination because engineering leaders and their teams have to constantly juggle competing operational priorities that the business doesn’t always have full visibility into:
- Dropping everything to squash that P0 bug
- Deciding to pay down tech debt vs. working on new features
- Shifting focus to a new customer enhancement based on product’s recommendation
But delivering operational excellence–represented by the to-do list outlined above–is no longer the only concern for engineering leaders. Now they are uniquely responsible for business outcomes that extend well beyond projecting feature delivery dates (though that is still a crucial part of the job).
Because engineering is now–rightly so–looked at as a key driver of business outcomes, leaders need to think about:
- If project resourcing is in-line with business priorities
- How to control costs and invest their limited resources wisely
- Maintaining operational excellence to enable other areas of the business to succeed
In this blog we’re going to explore how to answer those two (plus five more) questions correctly, every time they’re asked.
Current standard operating procedure for answering the big questions
For most leaders, knowing when you’ll deliver the new feature or if you can deliver it sooner is a matter of experience, intuition, (subpar) reporting, manual work, and lots of time spent in meetings–with a little bit of luck thrown in.
These tactics often yield mixed results (and that’s putting it kindly). In fact, leaders are unfortunately wrong more than they’re right (industry average <50%) when it comes to project forecasting and predictable delivery.
But why?
3 Reasons Why Accurate Project Forecasting is REALLY Hard
Tooling shortcomings
PM tools like Jira are great for product teams but not for engineering, and reporting has always been one of their biggest shortcomings. Not to mention the data hygiene consideration or the fact that building custom reports with the information that you really need requires time, effort, and a significant amount of PM tool-specific subject matter expertise (like knowing JQL).
Git and CI/CD tools are developer-first. This is a step in the right direction for engineering teams as the metrics they provide are more accurate. But they are overly code-focused and don’t really provide any insight beyond diffs, commits, merges, and deploys. While this information can be helpful, it’s not going to tell you:
- If your team can take on more work
- If added/unplanned work is bogging you down
- If your team is executing against planned activities
And you’re still contending with siloed reporting rather than a holistic picture–so manually correlating the sources and extracting some kind of meaning is a prerequisite.
Team status updates
Talking to your team leads and managers about the questions your CEO asked may work better than looking at tools, unfortunately it’s time-intensive. Not only are you slowing the project down more by introducing more meetings, but you’re only going to get the information the team lead has. Unless they’ve got a secret source of truth stored outside the tools above (which would defeat the purpose of having them) any information they provide will be anecdotal, subjective and often incomplete.
Lack of leading indicators
When engineering metrics are involved it’s easy to get tunnel vision. Story points, DORA, velocity, these metrics have been the yardstick for success in engineering for a long time. But they don’t net out to the ability to answer business-focused questions of engineering–at least not on their own. To understand these relationships and predict delivery accurately, you need to see leading indicators and more holistic metrics such as Planning Accuracy and Team Capacity Accuracy (more on this later).
5 “Small” questions that need answers before you can forecast delivery
To accurately forecast project delivery and/or determine if timelines can be moved up, there are variables that must be solved for (questions that must be answered). These considerations are the connective tissue between leading indicators (more on this in a bit) and results the business cares about:
1. How well are we scoping our iterations and executing on priorities?
You and your teams spend a significant amount of time and effort selecting your sprint candidates, prioritizing needs, planning the iteration, and assigning work before getting started. Planning Accuracy tells you how well you’re executing on those plans and whether there are things getting in your way.
2. Can we take on more work?
Many aspects of engineering can feel like a balancing act. Balancing the need to deliver on promises, keep your team engaged, and avoid burning them out is a great example. Capacity Accuracy tells you whether your team is taking on the right amount of work (and if it’s possible for them to take on more).
3. Is added or unplanned work bogging us down?
Most leaders know that engineering plans are often a fluid concept–someone needs a hotfix, strategies evolve, a high priority customer needs something urgently, etc. That’s why–especially when attempting to forecast project delivery–having a breakdown of unplanned or out-of-scope work is extremely helpful. Delivery breakdown shows you what project teams got done, what was added (and completed) mid-sprint, and what they didn’t get to.
4. Do we have enough resources to meet our goals/deadlines?
Fulfilling the business side of the dual mandate means ensuring you’ve got enough of the right people working on the right projects. If there’s a project deadline that needs to be met because of a planned release (like a really exciting Spring Launch 😉) you’ll want to prioritize that project. You also need to keep an eye on how those decisions impact your (likely flat or shrinking) budget as well as your other priority projects–engineering is a balancing act, after all. Resource Allocation shows you how you’re spending your team’s time and puts it in terms of real currency.
5. Are we balancing our investments correctly–innovation vs. keeping the lights on?
In a perfect world, engineering teams would only ever work on solving problems by developing new, exciting features. But the reality is teams have a lot of plates spinning–like paying down tech debt, refactoring code to avoid said tech debt in the future, or squashing bugs. Addressing all of those “other” concerns and balancing priorities is what makes new development possible. Breaking down work by issue type helps you get more detail into how teams spend their cycles and ensure they’re using their time wisely.
“But what about Velocity?!?”
One metric–velocity–has historically been used to try to answer some of these questions. But it’s a limited metric that can do more harm than good. Capacity has been used to try to answer others. And using them together has been tried as well. All these approaches are regarded as less than stellar.
Answer Questions and Forecast Projects With Accuracy Scores
Accuracy scores provide a ton of insight into a team’s ability to:
- Scope iterations and projects
- Execute on stated priorities and goals
- Adapt to the ebb and flow of project work
Planning Accuracy measures planned and completed work as a ratio of planned work. In a single “accuracy score” you’ll know if your teams are scoping iterations well, whether they’re completing their tasks, and how unplanned work is affecting execution.
Capacity Accuracy–one of the latest enhancements to LinearB–tells you if your teams are taking on the “correct” amount of work. It measures all completed (planned and unplanned) work as a ratio of planned work.
Take a look at the table below to see what accuracy scores can tell you.
In rare cases teams may have too high Planning Accuracy (>95%) and too High Capacity Accuracy (>115%). That means they aren’t taking on enough planned work–which should be a priority for improvement initiatives.
A note on Leading vs. Lagging Indicators
You may have noticed that all the metrics above are lagging indicators. One of the first things to do when trying to accurately forecast project delivery and become more predictable is to see, understand, and work to improve leading indicators: PR size, code churn, phases of cycle time, investment profile, and resource allocation.
Take PR size as an example: If your teams are consistently struggling with breaking work down into small, manageable chunks–a skill that requires a lot of creativity and is difficult to master, to be sure–they will produce large PRs. Large PRs take longer to be picked up for review, take longer to review, increase the risk of introducing bugs, and if/when something goes wrong, affect a team’s response time as there is more code to parse to find the issue.
Taken all together, leading indicators inform lagging indicators–Planning Accuracy and Capacity Accuracy.
Bringing it all together
Assuming your PRs are small, your cycles are fast, you’re working on the “right” things, and your projects are resourced appropriately, determining if you’ll meet a deadline comes down to accuracy scores.
Strong Planning Accuracy over several iterations is a fantastic indicator of delivery predictability. Capacity Accuracy adds another signal by quantifying team adaptability and illustrates the effect unplanned added work can have on delivery timelines and overall risk profile.
Together, these two metrics paint the most accurate picture of a team’s ability to execute and accommodate fluctuations that can–and do–happen during a sprint. Leaders can see these metrics on a wide timescale (months, quarters, or years) to identify overall trends or they can drill down into individual iterations to pinpoint the issues causing potential dips in delivery performance. And with simple, prescriptive guidance on scores, the next move is always clear.
In addition to execution ability, accuracy scores in LinearB, when taken with Resource Allocation and a team’s active days, can be used to surface other insights like:
- A need for additional resources on priority projects–especially if scope creep or shifting priorities are common occurrences.
- If teams have consistently low (or overly high) Capacity Accuracy scores, changes likely need to be made–Low scores mean too much work and could be an indicator of burnout. Overly high scores likely point to under resourced projects or unengaged team members.
Accuracy scores–both Planning Accuracy and Capacity Accuracy–are available to customers on Business and Enterprise subscriptions of LinearB.
You can get started with a free LinearB account today. Or schedule a demo to see accuracy scores in action.
