The Path to Engineering Operational Excellence

<prelude> One of the things I really like about Ben Horowitz’s books, (planning to read the new one soon), apart from the part that they’re awesome, is their ‘soundtrack’. If you want a soundtrack for this blog post, give a listen here or below. </prelude>

I love to develop and build products that impact customers’ lives and work; leading teams to pursue that end. This passion is what has driven me to co-found LinearB with my friend Dan Lines, who tells me that I’m well overdue to write a blog post. 

A core value of mine is that every improvement starts on day 1. This is true with personal improvement and equally applies to leading a team.  I’ve seen this over and over again whether it’s about a microservice optimization, self-improvement or scaling a team: There is usually a simple thing you can do that will improve your situation significantly (especially if you’re at the beginning of the optimization phases). Half the time, we don’t take that simple step because we are not aware of it and did not take the time to think about it. It’s not because we’re not smart, it’s just because this is how our brain works. We’re programmed to choose the path of least resistance or at least what seems to be that path at the beginning. The further we continue down a path, the harder it becomes to change our ways. 

Surprisingly enough, the other 50% of the time that we do not do simple stuff that will help us improve is not because we’re not aware of it, but that we’re too embarrassed to do it. We think it looks so basic and it’s been too long to start doing that simple thing now. I say, if you want to improve and be effective do it! You won’t look stupid, trust me, if anything you’ll look brilliant.

//Side Note: The Golden State Warriors combined analytics with good fundamentals to build a dynasty. Check it out.

Okay, that’s a pretty good opening to build my excuse for why I did not start writing content until now (because I was busy in other aspects of our organization). I think I’m forgiven now, so I’ll just roll up my sleeves and jump to it. Oh yeah, one last thing that’s important for me to mention is that English is not my native language so if something looks weird to you in my phrasing or my world of metaphors, it is what it is.

We’re in the business of helping customers build elite engineering organizations. We’ve built a platform that will help engineering teams at all kinds of companies – from ‘old’ industry companies that are going through digitization to sharp up-to-date SaaS companies – improve their quality and pipeline and deliver more value, and not less important: enjoy the scaling and improvement ride while they do it. Sounds ambitious doesn’t it? To try and tackle a problem that it’s not that easy to solve? I mean, there were attempts in the past to solve it and we’ll discuss in a future post (fingers crossed) why they missed IMHO. But there are there are 3 reasons why we think it’s a super relevant problem to solve:

1. It is an interesting problem with a need.

Engineering teams are our passion, I’ve built and led them my entire career. When you have an elite team that executes efficiently, as small as it may be, you can get huge things done. Unproportionately. Things that leave more than a ‘dent’ on industries and fields. Running a software team is also expensive. When you look into SaaS companies budgets they will be the 2nd or 1st most expensive department (depends on your company’s lifecycle stage ), so it has to be managed and optimized and cannot operate on gut feelings.

2. It is a tough problem to solve.

Everybody knows what’s the one bottom line metric that defines a Sales organization performance, but with engineering orgs, it’s tough. You can’t just walk in and count commits or lines of code. Everybody that knows something knows it’s BS. How can you measure value delivery in a consistent way? How do you identify bottlenecks? How do you create a team dynamic that makes the sum greater than its parts and not vice versa? Just to add some additional difficulty,  different roles are interested in different levels of answers to these questions. For example if you’re a team leader you need to control your iterations and tasks, you don’t want context switches and WIP overloads, while if you’re an executive you need 3-4 supermetrics like the well known SaaS ones to bridge over the engineering-business language barrier. Here at LinearB, we’re on a journey to help find answers to these questions for everybody.

3. The time to start improving is now.

A. because otherwise my opening paragraph was just an excuse and B. because, seriously, the timing couldn’t be more perfect.

It’s Kairos!

The ancient Greeks had two words for time: chronos and kairos. You’ve probably heard of chronos. It is the root word for English words such as chronology and refers to chronological or sequential time. Kairos, however, signifies a proper or opportune time for action. It’s kairos to get started making your organization better. Why is now the time?

Whether you lead a company or lead a team of people that are trying to achieve a certain goal people will often ask you ‘why?’ and you should be ready to answer. I get this question a lot from customers/partners/ potential investors. What’s so special in this era that makes it the right time to run a data-driven software engineering team?  

Here’s my take on it: there are a couple of factors that got together in this time frame that makes it the right time to start doing so. I’ll try to cover them here in this post then maybe in the next post will dive a little deeper on the ‘What’ and ‘How’: What to measure and to look for? And not less important: How to improve?

Data Availability

If I’d start this with ‘cloud drive productivity’ it almost feels like a saying from the 80s considering the pace of events in 2019, but seriously if you think about it: Git systems, project management software, CI/CD pipeline products – they all live in the cloud now, which was not the consensus 5-10 years ago (or for some highly regulated organizations even now). Also, there’s a lot of maturity and understanding of the standard ways in which these systems operate (git is the de-facto SCM standard which wasn’t always the case) and that the power of these systems lies in their open architecture, connectivity, and APIs. So in a very simple way, it’s easier to connect to systems that participate in the software development lifecycle in a seamless non-interrupting way that does not require teams to change anything in their day to day behavior, and get metadata from them so it can be sanitized, cleaned and pushed through data analysis pipelines that are already 100% applicable in other industries. It provides great insights.

Industry Maturity

The software industry has matured a lot. If you’re a junior developer and you disagree, I would say that elementary stuff like unit tests, coverage and some kind of agile methodology to operate was something that only a few teams in maybe 10% of the companies did just 10 or 15 years ago. If you’re a senior developer and you don’t share my perception, I would say that it could be that today’s younger generation of developers are much more specialized in specific languages/fields and maybe (as a generalization) not all had the privilege to writing enterprise quality code throughout all the stack levels, but for the exact same reasons mentioned before, you probably can acknowledge the methodological starting point of every developer and the tools is much higher. 

Anyway, I’m actually not talking about how easy it is to for an individual contributor to start their journey. Rather, I’m focusing on the progress that has been achieved around how engineering teams work together to achieve a shared goal. There’s a lot more standardization on how you plan your work (Scrum, Kanban), how to produce code (PRs), review features, releases etc. We’ve really come a long way. 

Haven’t read this yet? You should.

In addition, it’s exciting to see that there are thought-leading industry researchers like DORA who analyze Software Delivery Execution and are starting to formalize what engineering operational excellence looks like. If you haven’t read it until now you have to!

A new generation of data-driven humans


//Side Note: Gene Kim, one of the authors of the report has written a novel (The Unicorn Project) about developers in the age of data that is in pre-order now along with his first novel, (The Phoenix Project)

There’s a lot that can be said about the younger generation of developers that are making their first steps as professionals. It’s totally outside the scope of this post, but one very positive phenomenon that I’ve learned while working with younger developers is that they are looking for reasons and want to understand why are we doing things in the way that we’re doing. That’s the main reason that there’s much more acceptance of data. Data wins debates. I also think that in general, they have higher capacity to digest data, being born into a world where it’s table stakes level. In some sense, it’s the most data-driven generation of developers. There’s much more acceptance to being data-driven, detect bottlenecks, track team, and individual statistics and improve the factors that need work.

Summary

To conclude I want to convey 2 equally important messages. 

The first is that the elite engineering teams will incorporate data and insights into all of their levels – operational day to day and retrospective data-driven reports at the iteration level, delivery pipeline bottleneck decision and investment profiles at management level, and top of mind supermetrics for executives. Teams that will not adopt these methods into their day-to-day will stay behind. 
 

The second is that I wrote my first post! 

Stay hungry,

Ori