In order for engineering teams to become elite in terms of performance and delivery time, they need to continuously focus on improving how they deliver software. At LinearB we call this engineering intelligence, which is essentially the sum of the methods, approaches, and technologies that help dev teams become elite-performing teams. I’d like to argue that if you only measure one thing in your engineering organization or on your specific development team it should be cycle time.
This post is going to dive into what cycle time is, what can adversely affect this metric, what you can do to improve your cycle time, and how to practically apply what you’ve learned.
Data-Driven Engineering Teams
Most organizations today are looking to be data-driven in all aspects of the business, but many don’t yet know how to approach this when it comes to engineering. It’s hard to know how your team is performing when it comes to the many metrics of software development. After years of managing engineering teams, and many different approaches to software development, I came to realize that there is one critical metric that encapsulates end-to-end performance. Cycle time.
Cycle time is the time it takes from the moment a coding project gets started to the time it takes to get in the hands of your customers.
The only problem with measuring cycle time is that engineers consider themselves artists, and it’s hard to measure the muse. But seriously, it’s difficult to measure engineers’ performance in the form of output, lines of code, or velocity. Every person is different, and so is every project. Comparing engineers to one another to establish performance is a culture killer. So what can we measure?
Cycle Time – Measuring Teams & Processes
Based on much of the research we have performed ourselves–alongside industry experts, such as the team at DORA, and specifically, Dr. Nicole Forsgren’s research that was published in the highly recommended book Accelerate–we recommend you move your focus from measuring engineers, to measuring the team, and ultimately to measuring the process.
This is achieved by understanding the processes that get code from an idea to a working product.
To be more specific, this means how long it takes your team to deliver a piece of work from the beginning to the end. That piece of work could be a bug fix, a small feature, a refactor, or really any piece of work that the team is able to push from beginning to end quickly.
Production is everything. Code that doesn’t make it to production is not something to be celebrated. Until code makes it over the finish line and starts deriving value for the company you have essentially gained nothing.
There’s a lot of empirical evidence that shows that if you focus on the process by way of cycle time, you can learn how to repeatedly deliver small chunks of work all the way through to production more rapidly. In the spirit of Kaizen, this will ultimately translate to big gains for your organization.
Ok, this is much easier said than done. How do we actually achieve this?
To improve our engineering process and overall performance, we need to start by looking at the main productivity killers and how to remove them. Let’s explore.
Productivity Killers in Your Processes
Let’s start with context switches.
Everyone knows that if you want to work on something that is creative and maybe even difficult, you want to be in the zone. But getting in the zone is incredibly challenging if you are working on too many projects at once. Each project has its own dependencies and stakeholders that are likely to interrupt your attention at critical moments of focus. It’s much easier to complete tasks to the best of your ability when you tackle one problem at a time. And give yourself the time to do so.
Another productivity killer is a “Work in Progress Culture.”
This is a very popular modus operandi in many engineering organizations. There are often too many “balls in the air”, and the focus is on starting to work on things instead of actually getting things done.
Context Switches + WIP Culture = Dead Value. Let’s see how we can change these.
The Anatomy of Cycle Time
Now that we understand what affects productivity and cycle time, let’s take a look at the segments that actually comprise cycle time. If you fine-tune any of these stages, it will be like adding significant RPM to your engineering engine.
Cycle time is broken down into coding, pickup, review, and then ultimately the time it takes to deploy to production. Let’s zoom in.
Coding Time
When it comes to coding, this is typically a single person with a codebase and a requirement to build into the system. Whether it’s fixing a bug, creating a fancy new feature, or refactoring existing code, essentially coding time is when an individual contributor proposes a change to the codebase; where the endgame is actually applying this change in the production systems. In order to get this change promoted to production with modern teams using git practices, this would come in the form of a pull or merge request for code review.
Pickup Time
Pickup is the first handoff in the process. This is where another person needs to come in and take a look at the code before it can make it through to the next phase, the actual review. Oftentimes, code changes wait in the coding-to-pickup phase substantially longer than they should before undergoing review. The time between actually finishing the coding process and getting a person to free up and have the attention to review the code is dead time in terms of value. In other words, it’s just waiting time.
Review Time
Once they start reviewing the code (the next phase in the process), this segment addresses everything that needs to be done in order to be able to promote this code to production. Oftentimes there are comments during this phase and even requests for entire rewrites. This might take us back to the first phase in the process, coding, and then we again need to go through the process of pickup through to review, before getting the reviewer to look at the code again.
Because there is more than one person in the process at this point, many times there are differences of opinion, and arguments regarding necessary changes. But eventually, the end goal is getting both the coder and the reviewer to a place where they believe the code is in a good enough state to be merged into the production codebase.
Time to Deploy
The deploy time phase is exactly what it sounds like. Except that in the context of cycle time, this means the time it takes from merging in the code until it’s actually in the hands of customers in production (not behind feature flags, staging, QA or Pre-Prod). While production can vary in many ways for different development teams, the key is that it’s now actually bringing value to your users. Remember the initial goal when we started working on this piece of code was to get it in the hands of our customers, and therefore, understanding the time it takes our organization to push things to production users is crucial.
Cycle Time FTW!
So how do we take our existing team and improve our cycle time so it becomes more predictable, faster, and achieve all the benefits of rapid code to deployment?
Each segment in the process–coding → pickup → review → deploy–has different dynamics that impact the time it takes to complete each segment. If we incrementally optimize each segment in the process, this will improve our overall cycle time and performance substantially.
Tips for Improving Coding Time
Tip #1 – Improve Requirements
Many times when an engineer receives a requirement for building a change into the system, it’s the quality of that requirement that impacts code time most of all. What commonly happens is that an engineer gets started with the coding, only to discover that the requirements are incomplete, or in some edge cases, not defined at all. Other times the solution is not well-thought through. This requires the developer to go back to product or the requirement owner for clarifications and changes.
This back and forth adversely impacts coding time and makes the time spent writing this change significantly longer.
So the first tip to decrease your cycle time is to speed up coding by making sure product is providing well-defined and thought out requirements before having a developer get started on the actual work.
Tip #2 – Chop Down the Work
The next crucial step for speeding up coding time is to chop down the work into smaller chunks. Instead of trying to combine multiple tasks or concerns into a single change or pull request, divide and conquer. This is a crucial skill for teams to learn if they want to deliver important production code fixes more rapidly. What this means practically, is when working on a feature, don’t fix a bug that happens to be in the code next to the feature you’re working on. If you’re working on refactoring code, don’t try to push in a new feature at the same time.
Your best bet is to split the work so that you can focus on getting the coding task done end-to-end and deployed to production before starting a new task.
Chopping down the work is perhaps the easiest and most productive way to decrease cycle time and avoid unnecessary code complexity.
Tip #3 – Refactor Hairy Code
There are many places in the codebase where it’s difficult to make changes, that are more error-prone, and ultimately will take more time to fix to ensure no mistakes are made. It could be a great investment to refactor these parts of the codebase and remove these hard parts in the code so that coding of the next feature becomes much shorter and easier.
All of these tips individually, but in aggregate even more so, enable the move from the coding phase to pickup to be much quicker.
Tips for Improving Pickup Time
Tip #1 Coordination and Communication
Pickup time is all about team coordination and communication. This segment is basically the waiting time for someone to start reviewing my code. So the better we and our teammates get at communication, the more this will improve.
Oftentimes improving communication at this phase just means making sure those who are needed for review are aware that things are waiting on them for pick up. There are technologies that you can implement that automatically alert the team about tasks that are waiting for pickup. Software Delivery Intelligence tools like LinearB provide this type of automation for free.
The importance of communication cannot be stressed enough, especially now when it’s more challenging due to our current remote or semi-remote reality.
Tip #2 – Deliberate Context Switches
Remember context switches? Here’s a good tip for combatting untimely context switches.
Another tip that significantly helps speed up pickup time is deliberate context switching. Developers are typically writing features, bug fixes, or refactoring, essentially working on their own code. Doing code reviews is by definition going to require a context switch.
So if a context switch is required to complete the pickup and review segments, why not be deliberate about it? Why not wait for an existing context switch that is already happening and decide to do some code reviews at that time?
This can be done by waiting until right after a meeting is finished, right before you start your day, or even when you come back from lunch. These are all great opportunities to take a look at the pickup queue and the awaiting PRs that need to be looked at before diving back into your own work, and getting back into the coding zone, when you don’t want to be interrupted.
Tips for Improving Review Time
The review phase is a little more complex because it includes coordination with multiple people. The coder and the reviewers need to be involved, and there could be multiple cycles before the review is complete and signed off on.
So improved review time is highly dependent on great coordination and communication among teams. On top of these that are essential to the first three segments are a few more tips that are specific to review time.
Tip #1 – Done is Better than Started
To really get better at review you want to instill a culture where done is much better than started. Again this comes back to defeating work-in-progress culture, a real productivity killer.
Like all of us, developers like to start working on cool stuff. We like that much better than actually completing tasks and getting them over the finish line into the hands of customers.
We continuously have to fight that tendency to start new things before completing tasks. Particularly as a team, celebrate getting stuff done, over the finish line, and into production before starting new things.
Tip #2 – Cultivate More Reviewers
Review is typically the place where things get stuck. Some teams need to add reviewers, and to make sure that more people can review other people’s codes so that reviewers are not the bottleneck. Having a high review time can signal that you have a shortage of reviewers. So make sure to constantly be growing your engineers and increasing the pool of capable reviewers, so this is not the place that your process breaks down.
Tip #3 – Better Design Upfront
Other times the review will take a long time because the code needs a rewrite or because the design was not well thought through upfront. Good design, done early, will smooth the way for the rest of the process.
One suggestion you can try is to find other ways to vet your coding directions before you invest time in the actual coding work. This way, you won’t find yourself about to complete the coding task, only to receive a very critical review that will require a lot of rework to the code.
Tips for Improving Deploy Time
The last handoff in the cycle time process is deployment.
Tip #1 – CI/CD
Many times deployment time is out of the hands of the direct engineering team, but this metric can be significantly improved through investment in CI/CD automation. As we move to more automated deployment models, we need to work to gain as much coverage of the deployment process into CI/CD processes, and this will reduce friction in the process.
Tip #2 – Great Testing Practices
Another way to improve deployment time is by investing in your tests. Having tests that are more reliable and more comprehensive, will make you more comfortable with deploying more often. Great testing coverage that you can depend on will significantly reduce deployment time.
Tip #3 – Smaller Deployments
Having smaller pieces of work highly reduces the risk of deploys and helps you to deploy faster and more often. This also ties back to the coding phase, by breaking down the work into smaller, more manageable tasks. This is true for deployment as well, when you are deploying a massive, complex piece of code, it will require greater testing and quality assurance phases. The smaller the code change, the quicker it can be deployed.
The TL;DR on Cycle Time
Elite performing teams are those that have really cracked cycle time and are able to continuously push value through the coding to deployment cycle. The benefits an organization can achieve from improved cycle time are enormous.
- Shorter Learning Cycles
- Decreased Delivery Risk
- Greater Productivity Overall
When you deliver code to your customers more rapidly, the feedback loops are shorter as well. This means you can understand the value of the features you are delivering to your users and continuously improve their experience, a critical skill for nascent startups.
When you chop down the work from coding through to deployment, this dramatically reduces the risk of delivery with each change. By not creating huge bundles that are an integration hell, you gain the benefits of confidence with releasing more regularly.
Finally, progress can be shortened from weeks to days with fewer context switches, thereby increasing organizational productivity overall. The truly great thing about cycle time is that it can actually be improved pretty quickly. Based on data from hundreds of engineering teams, we have measured improvements of 20-30% just in the first month, and as much as 75% after one quarter when optimizing for the right things.
Finally, metrics or measurements are just the first step, but a great first step. If you really want to get great at software delivery, it’s not enough.
With visibility through metrics, you’ll find that most of your code is okay, and discover a small few changes that take much longer, adversely impacting your cycle time.
Understanding where those problems are and being able to isolate them will enable you to move from metrics to insights.
The next move is to actually take insights to action by enabling the team to interact with the issue as it’s happening…before it becomes a much larger (and harder to solve) issue. A good example of taking a metric insight to action is with the pickup phase. If we see a PR waiting too long for pickup, we need to alert the team and move a metric from insight to action.
By applying the tips I’ve laid out for you today, your team will be able to reduce their cycle time and push value to your customer much faster. Some might even start describing you as elite.
