The wars of the future will be fought with software and system architecture, as much as any other weapon. That’s why the US armed forces needed to start applying modern software methodologies at scale. The beginning of this incredible digital transformation began in 2017 when the US Air Force started a project called Kessel Run. Leading the project then, and now Chief of Platform of over 400 Kessel Run developers is my guest, Adam Furtado. 

Translating engineering to executives is a regular challenge for many of us, but I can almost guarantee translating engineering to military officials is way harder. When he’s not explaining why DevOps matters to his higher ups, Adam spends his time scaling Kessel Run operations by building systems of systems. Everything he has learned during his journey from 5 developers to 400+ is not something you want to miss.

In this episode of Dev Interrupted we discuss:

Episode Recap

Dan Lines: Yeah, I’m super excited to have you on the pod today. You know, I of course I looked you up. You have an incredible, incredible background which we’re going to get into here.

Dan Lines: And you know what? Let’s just start right there.

Adam Furtado: Can you give us kind of an introduction in history of Kessel Run?

Adam Furtado: Yeah, yeah, for sure. So Kessel Run is an Air Force organization that delivers a wide variety of mission capabilities to our war fighters that are around the world utilizing industry best practices around DevOps and Agile and UX and then what have you. It was kicked off about four years ago as a way to prove that the Department of Defense didn’t have to be terrible at building and delivering software, regardless of being within the world’s largest bureaucracy. So the is interesting in that we chose it after Han Solo’s same smuggling route and Star Wars, obviously, because we knew that we had to smuggle this new way of thinking into the DOD. This was kind of new for the way the DOD traditionally gets software or gets capability to users. Right. And what was happening in pocket’s Defense Digital Service and the GSA was doing some pockets of kind of modern software work, but it wasn’t really happening at the scale that we needed. So we attempted to do that by creating kind of a startup like culture inside a traditional, stodgy, kind of slow moving behemoth that is the federal government, and tried to move more quickly with continuous delivery and getting capabilities to users who needed them and have been kind of not getting them previous to Kessel Run, previous to getting into the kind of technology world. I was active duty Air Force for about eight years. And I can still remember, like driving to my car with my Bluetooth connected and like scrolling Twitter on the way into my office and having my phone in like a cubbyholes somewhere. And you walk into an office and you’re really transported to like nineteen seventy eight. It’s really hard to connect with people and send emails and communicate and all of that. And just the infrastructure in the way that we work within government is so antiquated that there’s just a real need to move it forward in a real way. So Kessel Run was our attempt to help start or at least put some more momentum behind that movement to get our users the tools they actually need to do their jobs in a really complex, important environment.

Dan Lines: Oh, man, that is so cool. First of all, thank you so much for your service. That’s amazing.

Dan Lines: And second of all, you know, one thing that we talk a lot about at LinearB is around translating engineering practices and what matters to a good engineering team to kind of those non-technical folks, sometimes that usually that be on the business side.

Dan Lines: In your case, this might be in the Air Force or the government.

Dan Lines: Did you run into any challenges within translation, you know, where you would want to be with Kessel Run or convincing officials that type of stuff? Can you talk about that?

Adam Furtado: I think you’re describing my job and it feels often like what I often feel like. I live in two completely different worlds and I think a lot of engineering and product leaders and big companies, especially enterprise size companies, probably feel this way. One of those jobs is leading product teams in trying to solve for exceptionally complex problems on the technical side of that. The problem is like our production environments are unclassified systems. So you can imagine the security implications and technical informations around, you know, cloud implementation or tooling, availability and SaaS tools and things like that. GRC complications interviewed in the bureaucracy of that size. But even on the product side, like it’s great. Our product problem is immense. Right. We’re solving for this, you know, potential future war, the tools that we make, our capabilities that allow or fighters to are deployed around the world to more efficiently do their jobs around what we call amassing air power for the Air Force in order to win a war. Potentially, hopefully we never get there. We don’t need to use it. But we’re preparing for this kind of future potential conflict that nobody’s ever experienced. Obviously, nobody can really describe our users have never experienced. Sometimes I like to think of it as like imagine you were preparing for a Black Friday event, but you didn’t know when Black Friday was, nor if it would ever happen. It’s like we were preparing for this, like, you know, synthetic future user that could potentially use our stuff.

Adam Furtado: It’s like this really hard problem. So imagine a big enterprise organization trying to solve for that complexity. But then my other job is trying to navigate the business people and the people who have been within the Department of Defense for 30 or 40 years and have lived this life of the new kind of new thing coming and telling them it’s going to change the world and it never working out because the government has been so poor at delivering needed capability to users for so long. Even in our specific problem, we’re trying to solve the government’s invested. Over a billion dollars in trying to solve it over the last 30 years with nothing really to show for it, so we’re trying to show that, OK, I understand that you’ve been burned a lot of a lot of times now. Let me explain to you why doing things in a more agile way and using deadbolts actually reduces risk for you and gives us more of an opportunity and more of a chance to be successful. Because in reality, the problem we have is we go and talk about like, you know, deployment frequency is going to buy down risk for us. That sounds counterintuitive to everybody in the world, particularly in a military environment where they’re like, what do you mean, change is scary? I don’t change stuff like. So we’re having this kind of counterintuitive conversations around why moving into this way of working is less risk and increases our chance of success when it seems obvious that’s a huge challenge for us.

Adam Furtado: So we’re basically doing, you know, four plus years of evangelism. Right. While we were kind of building all the tools and modernizing all these massive legacy complex systems. The other problem we have is a military swaps people out constantly so often where, you know, we finally get this relationship built and we’ve convinced somebody this is the way to go and then they’re moving to a new job. Somebody else comes in, we’re starting all over again. So but it’s important. And I think every kind of organization gets to a point where, you know, I often start to feel in those sorts of conversations like a guy like Charlie Brown, teacher effect, a great Adam is telling us about Agile again, the kind of zone out you define champions inside the business to do that work for you. So it’s not always coming from you and your perspective, but from their own perspective, a little bit more kind of empathy and that kind of thing. So, yeah, it’s really hard. We’re trying to find places where we can convince the right people in the right places that this is the right risk to take and the investment needs to come behind that in order to really move the needle at the scale that we’re hoping.

Dan Lines: Well, it is really hard, right? And our CEO, Ori, actually explained it in a similar way. He said, you know, I always feel like I’m the translator between two different worlds. These two different worlds don’t know how to talk to each other. If you don’t know how to talk to each other, it’s hard to influence people if you can’t influence it’s hard to make change because it’s exhausting.

Adam Furtado: Right for people and our position, we’re like we’re just trying to do the job and we just need some runway to do it and some space to do it can be exhausting. Like start from scratch, like, OK, OK, this is software. Here’s how code works. You know, you’re starting from that from your foundations.

Dan Lines: Now, on the flip side, do you have any story that that you could tell where you tried to translate and it totally did not work or, you know, you had a failure and any any tip, you printed it to the audience of, like, what not to do.

Adam Furtado: I think that maybe the biggest learning or the thing that comes to mind is early on in in Kessel Run, once we start with five of us, there are about five people that tend to build an application. We actually so the government doesn’t have organic software engineers or not. Many of them at least we haven’t really prioritized outsource everything, at least historically. So went out and found active duty software engineers who were scattered around the Air Force and brought them in and built. This initial application that we built is called Jigsaw to help us optimize in air refueling. So it’s like kind of a crazy process you think about. It is just like tankers that sit in the air and fighter jets do all their things and like in, you know, do their missions and they come back, they get refueled in the mid and mid air, do more missions. And you want to optimize everything around that, the airspace, the altitude, the timing, all those things. So we built an application to help optimize for that. And after about a year saving twelve point eight million dollars a month in fuel savings.

Adam Furtado: So that was a really good success story. Right.

Adam Furtado: And we had a few of those, like these small pockets of small balance teams that we like unleashed and empowered to go out and solve problems. I do think we learned over time that we missed a bit of a transition period where we were like really zeroing in on the second local maximum that we really needed to kind of think more holistically to optimize for the whole, if you will. And I think that was a bit of a learning for us to make sure that we are constantly evaluating the entirety of the system and the value to the value systems we were trying to affect. So I think we we missed a bit of a transition and what that looks like from moving to small power teams to how you can use that same mentality and methodology when building this large scale system of systems that’s needed to solve like a massive enterprise problem. I think that was.

Dan Lines: Yeah, well, yeah. Let’s talk about that a little bit. So it’s my understanding that you definitely did a good enough job in the trans translation with kind of the non tech folks that you did start with a team.I think you mentioned maybe five engineers to start kind of proving out your ideas.And I think you had some small wins there, some nice successes, but they might have been a little bit siloed.And maybe you have this concept of system of systems and maybe finding out you have a larger underlying issue that you need to solve, maybe for the entire organization that in just some some of these silos. Can you talk to us a little bit about what you experience there?

Adam Furtado: Sure, yeah, so the problem we were trying to solve is we had this isn’t they called er operation centers, they’re just these kind of these buildings, essentially these organizations around the world in twenty two different sites that manage how we fight an air war, essentially. So imagine everything from strategy to planning to tasking aircraft to do certain things, to keeping an eye on how that’s going in real time, assessing how we did planning the next day. It’s this constant cycle. And all of that was, you know, there’s hardware on each of these sites with a bunch of integrated software from various third party sources that have been in. It’s essentially the same state for the most part for about 20 to 30 years. That’s the problem we’re trying to solve. So we we knew that using God’s law in history that we needed to start small in order to make this work. We couldn’t just have a big bang approach to replace this entire system with that by chipping away at some core parts of the system from a user functionality perspective. And we had we had success there, but we started to see places where we realized we weren’t really affecting the larger whole that we cared about.

Adam Furtado: One example is back to that kind of tanker planner application. So the efficiencies we found is about five users that we’re doing like an eight hour manual process every day to plan the day’s worth of in air refueling, if you will. So we released this piece of software and we went back to measure the outcomes, right. So we had the software. That’s a hypothesis. And we think we’re going to be able to affect human behavior in a particular way. We go back and measure that. We learn that they were able to have the three people do the same job in three hours. So we’re saving all this time is a huge win right now. We’ve got all this time to do other war fighting stuff. So we go back. We’re like, cool, what’s the other war fighting stuff you guys are doing? And they’re like, what do you mean? We’re just like leaving earlier, playing more video games or that kind of shit library, just like we optimized for the wrong thing because that wasn’t that. It was making their lives easier. It wasn’t optimizing for this like bigger problem.

Dan Lines: We were have more video games, but not solving the bigger problem. Yeah. Which like I feel is good. Yeah. Yeah. I like that those guys are having a better time. Like that’s good. It’s not bad.

Adam Furtado: But anyway it got us thinking around like OK, how do we think about this holistically. How do we find ways to utilize our resources effectively to go solve the right problems next. And portfolio management in there and all of these things that were took us a long time to figure out. We’re still figuring out in a lot of ways. So I think that part was a particular struggle for us and in that it led to some other kind of intended consequences when one of my bigger regrets, I think, is that we got into this place of like, OK, we’re a year and a half in. We have these early wins. Right. We set up teams fast and we have these kind of early success stories. And now all of a sudden the stakeholders like what have you done for me lately? Kind of thing. And like we’re doing this. No, this big, big kind of problem. So we started kind of like going after or trying to get a couple of short term wins in order to keep that runway going. And we in reality did that at the expense of continuing to invest in the enablement of our staff and the structure and scaffolding needed to make the the inertia work for us, if you will. So early on, when there’s thirty or forty people, you can, like, navigate principles and culture and values based on being able to, like, just be around enough and navigate the limited communication pass.

Adam Furtado: You have to do it at a few hundred people to a thousand people at thirteen hundred right now, like you have to have invested and built that scaffolding and make it very obvious what your principles and values and philosophies are so that it’s obvious for people who come in, because right now we’re at a place where we’re bringing in people from from industry, which is like a really awesome achievement, because traditionally people who work for the government, if you had real talent and you wanted to be able to use cutting edge technologies and all that, like the government wasn’t the place you go to do that. It is now, right, there’s an opportunity for you to use the newest technology and also, you know, work on. Air Force fighter jets pretty cool now, but we wanted to make sure that, like when all of those different kind of types of people come in into this environment, we need to do a better job of investing in what Kessler Run was. So that like it was a little bit more stable long term. Now we’re going to do that work now, which is a bit of a disappointment for me.

Dan Lines: Yeah, I mean, you brought up a lot of a lot of good points there, and, you know, what we’ve seen from the LinearB community, a lot of the users of LinearB is is what we call scale ups. So these are kind of nice cutting edge software companies, but they’re looking to scale. Right? I have 50 developers and I’m going to go into hundreds now. And even in my own experience, you know, when I started to get you know, I was a VP of engineering, I started to do things like continuous integration and continuous deployment. I was able to do that starting with one team, OK, I get one team doing it. But it’s a whole different world to now do that for all teams or enable all teams. And that’s some of what our customer base is using LinearB for. Can I see for every single team who has followed that case, who doesn’t have bottlenecks, that type of thing. And of course, nothing’s easy.

Dan Lines: Even doing it in one siloed team is difficult, but I think it’s even more difficult to scale it out now. I don’t know how far along you are in that scale journey, but could you talk a little bit about what are some of the tactical steps that have worked for you in that scale of systems to empower all teams and also, of course, things that didn’t work?

Adam Furtado: You know, some of the things that we’ve tried have worked in pockets or to a degree, and we’re constantly evaluating and learning and growing and all of those things. I think the things that you mentioned were some of the things that we realized later to to be honest, none of this is right or wrong. But early on, we had a smaller team that wasn’t thinking about things like slow. I just like it wasn’t top of mind for me. I could, like, basically visualize the work just by being around and like it was just much more consumable for my brain at scale all of a sudden, like, I actually didn’t really know what the problem was that I didn’t I didn’t I didn’t recognize it as a flow problem or a theory of constraints problem until until later. But like, it’s what it was. So when we got to that point, we certainly started thinking about flow management and how we identify where these constraints are and start investing in those and, you know, using automation to our advantage and all of those things that came up, again, I think a little too late. What actually happened to us was as we built up, we grew into an organization that had all of our kind of application development and kind of one place. And that’s actually what I was leading all of our kind of software development. And we had our platform organization with our infrastructure SkyCity data and kind of a separate part of the organization. And like all the things that you would imagine happens with that. And Conway’s Law tell us what happened with that did happen.

Adam Furtado: Right. We all of a sudden there’s a bifurcation there and we have developers saying how shitty our operators are and vice versa. And the code sucks. That’s why it’s breaking and all that stuff. So we had to kind of come back together as an organization and think about our organizational design and bring that closer together. We’re still working through that. But as part of that, I actually moved over from app development to lead the platform and up side of things to help kind of bridge the divide a bit. So with that, we’ve done a few things. We are certainly focused on visualizing our work. So we consolidated our project management tooling down to try to do everything as much as we can within get lab as an example. So everything was able to be kind of visualize where we can. We’ve put a big focus on Observability and being able to kind of actually see what’s going on in our system in an easier way so we can react to it. On the kind of management side, we have implemented growth boards internally to really have these kind of like times where we can kind of get these teams together to think about, like, OK, how how are we doing on value creation? Here is just the best way to spend our resources. Should we kind of evaluate or pivot in any of these places? That’s been pretty useful. Our OK, our frameworks getting better and better every every time. It’s taken us about two years to kind of get the hang of it. I think so.

Adam Furtado: I think we’re trying a lot of these different things. But I think it’s I think the thing that I really learned here is like there’s not like implementation. You’re going to do like nobody’s going to listen to this podcast and be like, oh, shit, I got to take this meeting. We’re going to put it here. We’re going to solve our problems. It’s like constant constantly talking about the things that are valuable to you from like a thematic perspective. So, like, we’ve really, really hit recently that like the four things we care about. This is for our internal services organization, customer centricity, developer productivity, flow management and employee engagement. And we’re making sure that our metrics tie back to those four kind of lines of effort for improvement. And ensuring that, like even down to the people who are managing our own infrastructure or managing our identity management or whatever, all recognize that they’re there and tied to this larger mission that we have to support our dev teams who are creating value at the end of our value systems. And I think that part is what is kind of keeping us moving. It’s like, you know, the government doesn’t pay well generally. Right. Particularly for like tech people. So what we have going for us is like it’s a really mission oriented group of people. It’s entirely made up of people who want to work somewhere that has like a global impact and makes real change. I think that that’s what’s kind of keeping us there. But it’s been a long road for sure, and I have all the scars to prove it.

Dan Lines: Well, thank you for for sharing sharing on that. And of course, you know, at LinearB, we know all about measuring. You call it flow measuring flow. But the software delivery pipeline. And I just wanted to point out one really key thing that you said, because I hear it from our community over and over again, and it has to do with the visibility and the observability. Yeah. And if we’re thinking about, you know, kind of taking a step one, like, OK, I know I’m going to scale. And the issue that I hear, by the way, over and over again is you scale out and then you realize that you need the visibility and it feels too late. And that’s what I hear, you know, from a VP of engineering after VP of Engineering. I wish when I was, you know, smaller, I had a team of like 20 engineers, 30 engineers that I had that pipeline visibility in place, because then when I started scaling out, I could see actually where are the bottlenecks, what’s what’s working, what’s not working when you wait and it’s, you know, too late, of course you can recover. But I hear over and over again, that’s like, oh, man, I wish I did this like a year or two ago because now I’m playing catch up and I’m blind.

Adam Furtado: Yeah, I think I think that’s that’s entirely right. Like we did have, you know, some of that even from the beginning, but like not the level that we needed. And you just kind of like you do wish you had a little bit more foresight to kind of get ahead of the problems that you were coming for sure.

Dan Lines: And so, you know, kind of coming to to the last section here that I want to ask you about, we’re we’re a fairly data driven pod and we’re also a very data driven company at LinearB, and my product, LinearB is a Software Delivery Intelligence platform. So, you know, we’re helping development teams measure the success of their dev process, provide that visibility and insight and where to improve. Can you talk about how Kessel Run, measure success and maybe what metrics your teams are tracking?

Adam Furtado: Sure, yeah, so I mentioned it briefly earlier that like the idea of. Mission effectiveness or those user outcomes is like this, it’s the right thing with super so hard. So we are constantly evaluating how we can best measure the outcomes that we’re having post delivery right from our user perspective. So that looks a little bit different for each team. So each team thinks about kind of the metrics that matter to them and kind of what they’re hoping to achieve and we’re trying to make sure that we are doing that. It is inherently hard, though. But other than that, customer satisfaction metrics for sure, our software delivery performance or our door metrics are top of mind for us. But our four, I assume, yet the door for, you know, with availability and those kind of things, employee engagement, employee happiness and that kind of thing. And then we also think about stakeholder satisfaction we’re in that. We’re in a reality also that like our funding and, you know, we have to get supported by champions outside of the organization. So we also kind of try to measure how satisfied they are with our delivery against their against their needs. So those are kind of the big things that mission effectiveness one gets a little bit difficult for us. And that’s like at my level to go to the team level. Our infrastructure teams are certainly measuring lots of different things that are important to them. Constantly moving. Right. I mean, we’ve been going down various routes from a measurement perspective. And it’s it’s just something you’ve got to keep at and make sure that you’re measuring the right things that are not driving perverse incentives and all that. So we’re constantly kind of chasing that a little bit. But I feel like we’re getting to a really nice point.

Dan Lines: So insightful, Adam. You know, as I usually say, there is no one metric to rule them all, but it’s kind of a balance of metrics and the relationship together, together.

So, wow, this has just been an incredible conversation. Adam, thank you so much for coming on the show today.