LinearB's software factory spends more time on the spec than on the code. VP of Engineering Ariel Illouz spent the last year moving his team of about 30 engineers from vibe coding to spec-driven development, starting with a single spec-writing skill and a rule that every story above three story points gets a spec. This week on Dev Interrupted, he and returning guest Ofer Affias, LinearB's Senior Director of Product, join Andrew to walk through a project Ariel puts at six months without AI that shipped in roughly four weeks, at least two of them spent on specs. Ofer explains why proving the impact meant measuring AI adopters as separate cohorts. They close the conversation on where the software factory goes next, as agents take over more of the loops humans run today.
Show Notes
- LinearB Metrics Builder: Metrics Builder in LinearB. The custom metrics builder Ariel's team built spec-first, a project he puts at six months without AI.
- LinearB AI benchmarks: The AI engineering productivity gap: how elite teams pull ahead in 2026.
- Connect with Ariel: LinkedIn
- Connect with Ofer: LinkedIn
Transcript
(Disclaimer: may contain unintentionally confusing, inaccurate and/or amusing transcription errors)
[00:00:00] Andrew Zigler: My guests today on Dev Interrupted are really special because they're in the zeitgeist of building the software that everyone else is using to understand the software that they're building. Today we're doing a double feature. I have Ariel Illouz the VP of engineering at LinearB, and he's run engineering here since 2019, and over the last year, his team of about 30 engineers have moved from, you know, ad hoc vibe coding sessions like everyone else's to a more deliberate spec driven way of building with AI, AI. And with him is Ofer Affias our senior director of product, and he leads all things AI innovation on the team here at LinearB. And both of them are constantly obsessed with addressing the question that you too are obsessed with, which is once you adopt AI, how do we know how it's impacting our employees, our products, our downstream impact?
[00:00:56] Andrew Zigler: Is it for the better? And how can we make things better over [00:01:00] time? Ariel and Ofer, we're really excited to host you here today. Welcome to Dev Interrupted
[00:01:05] Ariel Illouz: Thank you for having us. Thank you
[00:01:07] Ofer Affias: Thank you very much. Excited to be here
[00:01:10] Andrew Zigler: Yeah, we're super stoked to have you all. Uh, Ofer, I know you've been in many conversations on the show. And Ariel, we're really excited to have you here for the first time and feature some of your thoughts around spec-driven development. And I kind of wanna start there because, uh, many teams right now are experimenting with a form of what we are calling in the industry a spec-driven development.
[00:01:30] Andrew Zigler: That is starting with the idea and spending a lot of time as a team writing it down in a shared space until you have a really strong mutual understanding of it, and then executing that into code. It really reflects the new way that we make software now. So Ariel, how did your team start doing spec-driven development?
[00:01:48] Ariel Illouz: As AI, uh, went viral, right? And, uh, we started to hear, to hear the, the term like again, uh, spec driven development because this is like a [00:02:00] super old methodology that, uh, we used like, uh, I don't know, 15 years ago.
[00:02:05] Andrew Zigler: Hmm
[00:02:06] Ariel Illouz: And, uh, and then we said, "Yeah, there is something here. We need to, uh, see how we like pushing that, uh, uh, forward."
[00:02:16] Ariel Illouz: So we did our homework first. Our CEO and founder, Ori Keren, uh, put us in touch with another startup that had already rolled this out across their whole company. So we sat down with them, asked a lot of questions, interviewed their architect and, uh, senior developers, and took away a few good lessons before we started, uh, doing anything.
[00:02:41] Ariel Illouz: Then we started, we decided we have like a- an AI guild, uh, that, uh, you know, we are, gathering together like every, uh, second week. And, uh, we did the simple thing we wanted to do, just one skill that start with writing the spec [00:03:00] for the engineer. So that was it. This is, uh, uh, the only one skill that we begin with.
[00:03:07] Ariel Illouz: Uh, we didn't go... We didn't want to go with the big changes. We wanted the small and safe steps. This is how we started, and, um, and yeah, now the team is using this skill, which creating the, the spec doc itself. Uh, it's like interactive, uh, uh, thing. Uh, it's like, uh, getting a Jira ticket and then starting like, uh, some, conversation with the engineer, uh, regarding the ticket, uh, gathering some information, uh, and then eventually it creating the, the spec, uh, document
[00:03:46] Andrew Zigler: Okay. So what I'm hearing is a really s- is a really compact three-part playbook. So first off, you, you met with the team that had pre-established a practice and you, you extracted the learnings, the failures. You, you fast-forwarded [00:04:00] through a period of time of experimenting. You had another thing too, which is, uh, a guild, like some kind of employee group.
[00:04:07] Andrew Zigler: That's super critical. We talk about that here all the time with leaders on the show, about having a regular shared place, an ecosystem, just a sharing space for folks to collaborate is essential for getting any kind of momentum with something like this. And then lastly, not falling into the temptation of making like a Swiss Army knife of tools and skills and all of this stuff that we do.
[00:04:29] Andrew Zigler: We have one skill. This skill is like an interview-driven skill that pulls out of your head everything that needs to be in the document, and everyone goes around and puts more and more and refines it and refines it. That's the only skill we need. Because the real insight there is that if you have those three things, you can make every other skill, every other group, every other practice that you want if you all agree, which is the point. And so it becomes like a way of dividing up, like a [00:05:00] dividing up work so that everyone can co- contribute as well. So like how did that look once adoption started and you have this practice? Were there obvious candidates for we're gonna try spec-driven development on that for different types of work that y'all would do on the engineering team?
[00:05:19] Ariel Illouz: Yeah, so we are not, we, we, we, we have like a, a rule of thumbs. We have like, uh, something we started with. This is, you know, this is like a continuous improving. We l- we are learning all the time, and, uh, but we had to start it with something. So we said, uh, any story above three story points, excluding DevOps task and, uh, those kind, you know, like, uh, task you, you don't really need spec doc for that.
[00:05:49] Ariel Illouz: We said, "This is a must." This is not like a nice to have. We wanted to make sure that everyone understand this is like a big change we are going to push [00:06:00] to the engineering team, and it's not like, uh, something, you can decide yourself to do or not to do. This is everyone is doing that above three story points.
[00:06:10] Ariel Illouz: So we start to measure that. We also have KPIs for teams that measure that, across the board
[00:06:18] Ariel Illouz: and now is everyone, using the skill. We are getting, uh, on a weekly basis feedback. We are improving that, uh, improving the skill, uh, and, you know, pushing, uh, uh, like improvement, uh, continuously.
[00:06:34] Andrew Zigler: Nice. And, and so I, I kinda wanna, uh, open the floor to Ofer here as well to chime in about the experience of being like a product leader and thinking about the engineering team and the product team starting to converge, right, on this shared space of collaboration, the spec. And I'm, I'm wondering what has been the experience like as a product leader collaborating with engineering leaders, creating the, these kinds of specs.
[00:06:58] Andrew Zigler: Do you... H- how has that helped [00:07:00] your own process? How has it enriched the, the product? And like, what have you learned that maybe, um, others could learn from as well?
[00:07:06] Ofer Affias: I want to understand like the, the aspect of, um, uh, of the measurement and how you like prove the, how it w- w- how it looked from, from our v- our side. And I think that, um, when we, when we first started, we looked at the obvious, the classical, uh, measurements, um, of, uh, of the classical DORA and how it affects, um, uh, the cycle time. we, we wanted like, uh, to see, how much adoption of these tools affect these, uh, these KPIs. We weren't alone. By the way, we are looking at the industry. We saw this practice. Um, and we didn't see any, any, uh, any proof and what we, when we, uh, when we thought about it, we understood that, um, the, [00:08:00] these processes, makes the delivery, the coding, uh, faster, um, they don't affect the, the, the end-to-end delivery, which still has like human touches, like the review, like the delivery. Um, so it's capped. And, um, if you look at it, the, the highly, um, the, like the skilled developers, those that, uh, uh, the early adopters that adopt the A practices the more, they compensated on that more delivery, doing more at the same time. And when we looked into, uh, these KPIs, the merge frequency, we saw that there is, there is, there is a spike. And, uh, one interesting thing that, uh, when we, uh, I think, uh, we released, uh, LinearB released, earlier this year, the, AI benchmark report, we, we show it in the industry. You see that at a certain, at a certain inflection point when, when, when the models [00:09:00] became, uh, capable enough, and it was, um, at December, last year,
[00:09:07] Ofer Affias: you see a spike of, uh, of like the cohorts of, if you take the early adopters, th- those that use AI the most, they, um, more than double, almost triple their delivery rates. We see it also here. so I think this was like, uh, a learning point. We understood that not only the software do, like the practices changed, also how you measure it and how you, uh, the need to, to, to differentiate between the cohorts. You can't just like, uh, treat it like as a uniform, um, uh, number like we, uh, did before, which was more, like, uh, it was easier, but now you need to like, uh, define the cohorts and, and measure the right KPI. Um, and then we, we got the proof and, and [00:10:00] it, it supported like the transition I think the, uh, engineering did.
[00:10:05] Andrew Zigler: It definitely showed how the pressure points changed and how you definitely got a, a lot more output and frequency and capability that showed... Like, we, I remember when we did, uh, we'd had, had that webinar where we walked through these AI benchmarks at LinearB, and, uh, you could see so clearly the spike around December, and you can see really a gap start to abide between heavy AI practitioners and those that are still dabbling, and it really spoke to, like, the experimental opportunities that were available.
[00:10:39] Andrew Zigler: And really, what I, what I'm hearing too is, like, and what's been really interesting is that h- you know, we've talked so much about those benchmarks 'cause we aggregate them on an industry. It's been really fascinating to see how that's actually impacting the way people ship software. But it, it's also interesting to me to hear, pulling back the curtain, and being the team [00:11:00] behind that and making that possible and making...
[00:11:02] Andrew Zigler: And thinking about applying those same kinds of AI practices internally. There's a lot of insight that you can pull out of this data and learn from, and one of them being, like, the bottleneck of, of, of reviews, and we're gonna be talking more about that later as well. But you called out a really important thing, which is you just get a lot more merge frequency.
[00:11:22] Andrew Zigler: People are just generating more text and more code, more sometimes than they can keep in their own head. So you get a challenge of engineers to, trying to remember and understand everything they're writing and shipping. And then I think this then comes back to the power of the spec, because up until now, w- you know, code has always been, like, what we look to to know if something is working, to know the final result.
[00:11:51] Andrew Zigler: Code was crafted by a human, and when you wanted to know how something worked, you went back to the source control. Now, if you wanna know how [00:12:00] something works, the code is just as much of an artifact as the deployment, as the evals, as everything else. You have to go back to the engineer, or you have to, in this case, go back to the spec. So I, I'm curious in y'all's practices of developing these specs, like, what has that looked like, um, in terms of accumulating that, that information and keeping it, like, really effective? Since it's effectively now as important as source control, do you have any strategies that help you differentiate it from, like, a prompt, for example?
[00:12:35] Andrew Zigler: Like, how you think about collecting text over time?
[00:12:38] Ariel Illouz: So yeah, so first of all, you know, uh, there is no question here. Prompt is short, you never really know what you're going to get back, right? All the vibe coding. If you, if you, uh, think about it, all the time we spend on the back and forth with vibe coding, it's like f- I think like maybe two times, three times the time that, uh, we will [00:13:00] spend, uh, less, if we build, build a spec.
[00:13:04] Ariel Illouz: So there's no question about it. spec, spec driven development, this is, uh, how you go with AI when you have like this machine that produce all this code. So this is, uh, what we, uh, think about spec. This is a full picture, and we do not write it by hand. We have scale for that, and it pulls in our internal software design and all the constraints we work with.
[00:13:27] Ariel Illouz: So what comes out already knows how we build things. At the end, I think the time spent on back and forth, as I said, uh, is much, much longer than creating a full detailed, uh, spec. So we have a, a folder in, uh, Confluence, which we are accumulating all those spec documents which are frozen in time after you complete the, the feature, right?
[00:13:55] Ariel Illouz: And this is one spec, uh, methodology that, uh, we are, ongoing. [00:14:00] And we have another spec which is something that, uh, you know, describe a, a service. This is like a live spec that continuously change and update while we go and update the feature, update the page, update the service, whatever.
[00:14:16] Ariel Illouz: So we have two kind of, uh, uh, spec, uh, docs.
[00:14:20] Andrew Zigler: Oh yeah, that's interesting. It's like not all specs are, are equal and also the idea that like once a spec is shipped or the feature's shipped and it's out there, you lock in the spec. There's no
[00:14:32] Ariel Illouz: Yes.
[00:14:32] Andrew Zigler: You don't mutate it anymore
[00:14:34] Ariel Illouz: Yes
[00:14:35] Andrew Zigler: you treat them with the same level of deference as you would like source control when you would have live, something shipped and live and up.
[00:14:41] Andrew Zigler: You're not going to make a change to it without going through another process and in this case sounds like it's another spec. This would actually be a new spec that y'all would then go and work on. Um, Ofer, I'm curious like what your thoughts are on what makes for a good spec for yourself when you think about creating these kinds of new tools now in, in the kind of methodology [00:15:00] that Ariel's describing?
[00:15:01] Ofer Affias: So I think first of all, it's, uh, the process of the spec. It made, um, the ability to discuss, um, the feature, something one can do by himself. Uh, which is very interesting because, uh, before that, uh, if you wanted like to do, uh, a good, um, let's say design review, you had to, um, summon this committee of, uh, per feature.
[00:15:26] Andrew Zigler: Yeah
[00:15:27] Ofer Affias: was the Holy Grail. but it very... w- was very hard, like, to meet with this, um, standard. and it didn't allow-- It wasn't scalable, let's say. and now with the skill, the skill actually interrogate, uh, the developer, and the developer, uh, can s- can start thinking and bring these questions to the product if needed. all this thought process is taking a po- like, uh, is happening a way asy- in an [00:16:00] async way, in a way. Um, and moreover, it, it happens for every, almost every story, like Ariel said, like the three points rule. So if you look beforehand, the level of focus that we needed as a product to put on, on stories it's something we, we couldn't do. Now almost every story gets that attention. this is with, uh, you know, with the, the AI relentlessness, uh, that makes it, makes it like, uh, something affordable. Uh, it's consistent. Um, and for me, like, I think what clearly changed, I was used like that every, um, quarter, the main, feedback from engineering to the product team was some amount, big amount of the stories is not baked enough, is not prepared enough.
[00:16:53] Ofer Affias: Uh, and this led to A, B, C, D this last quarter, please do better. And, [00:17:00] and, uh, it went for, for years. It was like the state of the industry. And now we barely hear that, uh, feedback. Um, and this is for me, like a very big change.
[00:17:11] Andrew Zigler: Yeah. It's like, it's really what specs have opened up is not only just a way to collaborate as a team, but also a way to refine your ideas. Because you're rightfully calling out that, you know, summoning that product committee to all weigh in with their perspectives on something is the holy grail. That's what you want because that's how you're gonna get the richest feedback.
[00:17:30] Andrew Zigler: But it is, um, really hard to scale that inside of most engineering orgs. But going back to the original playbook, if y'all have a skill that interrogates, as you said, that really pulls all of that question out of you, you can just work with that skill on a daily basis and refine it and refine it. And then when you get to this point of, you know, are these...
[00:17:50] Andrew Zigler: You, you said are these story points, like are they baked enough? Has enough thought go into them? Are they ready enough to start moving through the queue? That then becomes maybe a more viable [00:18:00] question. I,
[00:18:00] Andrew Zigler: I
[00:18:00] Andrew Zigler: do wonder if, if you, when you take away that friction,
[00:18:05] Andrew Zigler: you these, pull away those, those, uh, things that were holding it back,
[00:18:09] Andrew Zigler: Is there a, an issue with noise? Like if ev- if any kind of idea can get refined and can get baked and look like it's a good idea to ship through, is there, is there then more necessity for like a really high level of taste and judgment of knowing, yes, we should build this, or no, we shouldn't? How
[00:18:28] Andrew Zigler: How has
[00:18:28] Andrew Zigler: like the proliferation of specs kind of like sharpened how you build?
[00:18:33] Ofer Affias: I didn't see any impact on, on that. Um, I think, the business intent is, uh, is already set most of the time. So, um, it's more about like, um... I, I think maybe it-
[00:18:49] Andrew Zigler: is a good idea once it's there. It's about making those ideas as good as possible
[00:18:54] Ofer Affias: Uh, yes. And, and I think that, um, uh, for me, like the [00:19:00] understanding that people are not thinking, uh, by themself, uh, in a natural way. Like most of people need collaboration, discussion to, um, go through like, a thought process. And in that way, the AI is great at like being the counterpart. specifically for your question, I think that, at least at the moment, this loop happens more in the engineering, loop, and maybe we'll discuss it later. I think it still didn't get like to the outer loop, uh, at that extent. Um, but I think once it, it will get to the product loop and, and the business loop. for sure there will be, uh, an effect on that like this, I, I, I believe.
[00:19:43] Andrew Zigler: Yeah. You're, you're right to say that it's like loops on loops be- and you have to start with that smallest, tightest loop before you get the product loop and the business loop. I'm curious for, Ariel, from your perspective, you know, like how did you get the team on board? Right now we've talked about, you know, this is the i- this is what we're going to [00:20:00] do.
[00:20:00] Andrew Zigler: These are the criteria, you know, above this number of story points you have to, to write this. Um, how do you then get everybody, uh, on the same page, excited about doing it, um, uh, solving any friction that comes up along the way? Just being realistic about how teams adopt, you know, what were things that maybe surprised you about that once you started?
[00:20:21] Ariel Illouz: We knew that it going to be hard, and, you know, first of all, I wanted to make sure that no one is writing code like manually. If it's like a story, like if it's like, I don't know, like a few lines, whatever, I think, you know, me personally, I'm writing code, and even like one line of change, I'm going to do that with AI.
[00:20:45] Ariel Illouz: This is me, but some, you know, some engineers prefer, I don't know, to, uh, keep in shape and, uh, and to write code
[00:20:54] Andrew Zigler: get their reps in, write a little bit of code every once in a while. I made fun of my friend the other [00:21:00] day because they use Claude to, uh, clone a git
[00:21:02] Ariel Illouz: Yeah
[00:21:02] Andrew Zigler: I was like, "I guess it's kind of the same."
[00:21:05] Ariel Illouz: Yeah. Yeah. So until now, when I'm, you know, when I'm seeing like commits without co-author of any AI tool, this is like, uh, it's, you know, it's makes me, I don't know, amazed. Um, I think, um, we have like, uh, you know, one or two engineers today that, uh, is keeping pushing manual code, but I think this is also going to, uh, disappear.
[00:21:34] Ariel Illouz: So we can say everyone, 95%, is not, you know, using AI to write code. This is one thing. So this is, I verified and I keep saying that in a, you know, in our meetings to everyone. Uh, and, uh, re- regarding the how we verify that everyone is using [00:22:00] the skill and, uh, and, it was like, uh, something we did.
[00:22:04] Ariel Illouz: This is not something, uh, we recommend you to do. This is something we want you to do, and we are going to, uh, check you. We're going to, you know, measure that. We're going to see which stories above three story points wasn't u- uh, we, we... You, you didn't use that, and we're gonna ask questions. So everyone is understanding that everything is changing, so they should adopt, and if they don't, uh, it's like, uh, it's going to be a problem.
[00:22:38] Ariel Illouz: I don't know how to say that. So yeah.
[00:22:42] Andrew Zigler: You set, you set a b- you set a bar. You, right? You're s- as you're more so setting a bar and saying that, um, this is what we do with this practice. If this practice isn't followed when we do this, we're going to scrutinize it because we're trying to understand what is holding you back from being able to make that jump.
[00:22:58] Andrew Zigler: Is it a tooling [00:23:00] problem? Is it a skill problem? Is it a context problem? Like, whatever it is, you want to be right there, you wanna identify it, and you wanna solve it. Um, because and the reality is, and as you're describing, most of the team is in this kind of loop now where they're, they are engaging in this, and it's totally flipped their craft
[00:23:18] Ariel Illouz: Yeah
[00:23:19] Andrew Zigler: But it, the, at the end of the day, they're still driving their craft because there's a great deal of control that you are still executing
[00:23:26] Ariel Illouz: Yes.
[00:23:26] Andrew Zigler: specs.
[00:23:27] Andrew Zigler: And so, like, I, I, I, I wanna take now the conversation and bridge this into talking about applying this methodology. We've talked about how we got there, how you start, what it was like to get the whole team adopted, what are the obvious benefits and, and, and drawbacks.
[00:23:43] Andrew Zigler: You know, Ariel, you, you, you talked about the loops, and we all talked about, like, how you have a smaller loop and a bigger loop. I think there's a really interesting conversation to be had about LinearB in general because our smaller loop, our engineering team, is inside of this really interesting other loop that we [00:24:00] track, you know, this, uh, the general SDLC of the industry, how everyone else is using AI to impact their tooling because that's what we build, is the proof and the tooling that they use to understand the software they ship.
[00:24:13] Andrew Zigler: It's like a meta conversation, and what's really interesting is that the practices we get inside, honestly, we get, we learn so much from the industry when we pair those things. So one of them that's been really interesting to see from most tools, and I, to our listeners, I know this is gonna be pretty familiar with them, is the apps that we use every day are just totally transforming. You don't go to an app anymore and log in and click a bunch of buttons and dashboards and just get something printed out at you. Instead, more often you're going to a place and you're having a conversation, or someone's showing up where you do all your work and having the conversation with you, and it has all of the data and knowledge from where it came from.
[00:24:54] Andrew Zigler: And I think LinearB is, is on that journey as well. I'm curious what it's been like to, [00:25:00] uh, kind of like power this kind of transformation inside of LinearB and, and how do you think about building the tooling to kinda make that possible?
[00:25:08] Ofer Affias: So I want to firstly give a context on, uh, on the challenge that we are, we are facing because, like all the software, organizations are having this, a, a challenge of transformation, of, uh, the change of the craft. But for LinearB that our customers are this software organization, it's like a double challenge. Um, and we look at it, um, uh, we, and we see like the development is going like through a drastic change, obviously. And, it's already went through a change. I just, just mentioned it that, um, uh, the, the, the measurements that you need to check are different, and, and we are looking forward, and it doesn't look, uh, that it stabilized. Like we understand as a product company [00:26:00] that, changes are, are, are, are coming, and, and, and, um, we'll probably have few more measurements we'll need to support in the product.
[00:26:11] Andrew Zigler: Yes
[00:26:12] Ofer Affias: and like you said, like also the, the how you consume it. So, I think like to give context, we, we, we had two high level goals when we, when we approached that.
[00:26:24] Ofer Affias: One, wanted to be able to support rapid adoption. So we, we wanted like to be able to define a new metric and connect it to, um, uh, a, a user value in less than a week. So for example, you want to measure something X, and you want to turn it into value to, uh, consume it somewhere in less than a week, the whole product loop.
[00:26:52] Ofer Affias: This is like one. And the other thing is that we understood that there are not only one surface now. Like it's not like the old [00:27:00] dashboards that you, you log in and monitor, like you said. There are multiple surfaces. The AI, might it be in the, in the platform, or maybe you're, you're preferring one, like your favorite, uh, Claude or Codex or one of these. Um, or, um, an API, like because we also see a surge of agentic processing of the same context. So I think to, to set a context, this is the challenge we faced when we started to think, uh, like, um, what we are doing next.
[00:27:37] Andrew Zigler: Yeah. I, I love how you called it a double challenge. It's a really, like, smart way of phrasing it about how ultimately your customers are trying to understand this same challenge while y'all are grappling with it internally as an
[00:27:50] Ariel Illouz: Yes
[00:27:51] Andrew Zigler: And, and so you're having this double sell. You as a, you in a, as a leader are always in this pivot between the execs and the engineers.
[00:27:59] Andrew Zigler: Now [00:28:00] you're in, like, another pivot between then also the industry you're selling to, and you're just, like, spinning around in your chair. So it sounds like honestly like, um, one of the most important things that you need, and Ofer, I love that you identified this, is a really short feedback loop. You have a theory, you have something you wanna track, you wanna pin something into your product, whether it's because you need to know it or because the customers need to know it or whoever in between. If you can't ship that and get that showing, showing value in a week, then you're dead in the water because you can't move fast enough. All of the other loops are gonna tear you apart. So, um, I, I think that becomes, like, a really, uh... It, it speaks to the challenge, I think, of shipping new things. Um, Ofer, you said also, like, API consumption's going up.
[00:28:44] Andrew Zigler: I think everyone's experiencing this across the industry. All companies are transforming into these consumption-driven services because that's where, uh, that's how, that's how folks want to consume, um, the, the information, the data, the context. Uh, a really [00:29:00] interesting thing too is that then, meeting the industry with where it needs, you, you maybe, you may, you, you discover new metrics you wanna track.
[00:29:07] Andrew Zigler: You create a new idea of how you prove success. But now you also have to create tooling to put that in someone's hands. Th- they have to be able to ask about something that six months ago, six weeks ago, they weren't even tracking or, or saying. So, like, what are some of the things, uh, maybe, uh, Ariel, if you wanna chime in about, like, capabilities that y'all have been building in the platform.
[00:29:30] Andrew Zigler: Thanks to your new spec driven development process, what have you been able to
[00:29:34] Ariel Illouz: Yeah.
[00:29:34] Andrew Zigler: folks?
[00:29:35] Ariel Illouz: So yeah, so we have the, uh, the custom metrics builder. Uh, this is like, a way, uh, for the customer to build any, uh, uh, metric, uh, uh, consolidated, composite, aggregated, whatever slice and dice you, what you want to do. and I think this is referring to what Ofer just, uh, said about [00:30:00] that, that we will need to provide some tool in order to provide those, uh, one week, you know, metric, uh, starting measuring something.
[00:30:11] Ariel Illouz: So this is already proving, like we are continuously pushing new metrics, new measurements to the custom metrics, uh, builder, providing this, uh, great rapid,
[00:30:23] Ariel Illouz: how?
[00:30:23] Ariel Illouz: ability to create those, uh, new metrics on a weekly basis. And we, we have built this, uh, feature in, uh, spec-driven development. This is was our big, uh, first big, uh, story that we, uh, did as a proof of concept.
[00:30:40] Ariel Illouz: And I can tell you this is like a six-month project in a w- world without AI. Six month. And with spec-driven development, we did it in, uh, less than month, less than one month.
[00:30:54] Andrew Zigler: So just kind of dramatically shortened,
[00:30:56] Ariel Illouz: dramatically short. Yes. Yes[00:31:00]
[00:31:00] Andrew Zigler: Did you find that because the, the bottleneck was about getting aligned or because once you were aligned, then the code was basically free because you could
[00:31:09] Ariel Illouz: Yes.
[00:31:10] Andrew Zigler: that you
[00:31:10] Ariel Illouz: Yeah.
[00:31:11] Andrew Zigler: uh, based on the idea?
[00:31:13] Ariel Illouz: Of the time, if I'm breaking down this, uh, four weeks, let's say four weeks, like, uh, at least, I don't know, like two weeks we, put on design and spec docs all over. We had like around 20 spec docs for different, uh, part of the system. This is a whole new database, ETLs, pi- a new pipeline, and, uh, like a real, uh, complex and challenging, uh, engineering project.
[00:31:42] Andrew Zigler: Yeah
[00:31:43] Ariel Illouz: And just after we had, like, this, you know, spec ready, we hit enter on the Claude, uh, you know, auto, uh, creating.
[00:31:54] Andrew Zigler: Yeah, and it, and it takes the spec off
[00:31:56] Ariel Illouz: And it takes...
[00:31:57] Andrew Zigler: puts it on the line and it just makes all the code[00:32:00]
[00:32:00] Ariel Illouz: Yeah.
[00:32:00] Andrew Zigler: plugs it into everything
[00:32:01] Ariel Illouz: And then
[00:32:02] Andrew Zigler: the rest of the work was.
[00:32:03] Ariel Illouz: Yeah. And then it start generating everything, and we had to, you know, verify all the components, integrations, and all of that. So it went really smooth, and this is was like an, you know, eye-opening for us that, uh, you know, this is the right direction and this is where we go.
[00:32:23] Andrew Zigler: Right. Ofer, I'm curious, like what is, what was a lesson for you in, in doing the work in this way, uh, in working a spec driven development as a product leader?
[00:32:33] Ofer Affias: Yeah. I think, um, I'm, I'm looking on it, uh, from the end result. Like, we shifted as a product organization from, um, um, qu- like quarterly deliveries, big deliveries, to weekly, biweekly deliveries. We call it beats. so it's, uh, high paced, high volume deliveries. I'm, I just like, uh, remember like, [00:33:00] um, uh, a week ago, one of, uh, the feature we discussed, uh, went from ideation um, to delivery actually in, in less than 24 hours. And, and I was amazed, like, because I think this, this amount of, uh, pace is something that was unimaginable before. And yeah, and, and the takeaway is I think that we are at like an, an exponential curve. Um, and you have to adapt. You have to adapt because it, um, if you are not adapting, uh, your competitor is adapting.
[00:33:38] Ofer Affias: Uh, and, uh, and, and the gap is, is widening in an exponential like, uh, speed. so it's either you adapt or, uh... It's, it's like, uh, it's, it's, it's you need it to survive. It's not like something, um, uh, nice to have.
[00:33:54] Andrew Zigler: to have exactly. It's, it's an expectation. I, I like that you, you know, called [00:34:00] out even getting to a point of doing something in a day. There's a very, a very, like a thought-provoking quote from Claire Vo, um, who's on Lenny's newsletter. She does like how AI segments and, uh, she's a really great leader about thinking about this stuff.
[00:34:14] Andrew Zigler: And something she said is a huge red flag for her for any like brand new AI native company person thing is if they can't do something in a day. The idea is that if you can get your loop down to a day, then like you're more po- like, it, it makes you extremely powerful in iterating. Um, and so people are trying to make those loops as tight as possible, and I think that there's a lot of ways that spec driven development helps us get there. Um, but also I'm sure too, like when you're doing it as a team, it's not always gonna be perfect. You know, we described this like magical rack of specs and you pull one off and then it goes down the conveyor belt, but you still have business owners, you still have reviewers, you still have a lot of disagreement that's still gonna happen once it's moving down, uh, the, the [00:35:00] treadmill, you know?
[00:35:00] Andrew Zigler: So I'm curious, like, uh, are there any places that you've seen where spec driven devel- development breaks down? I know we've talked about criteria around like story points, but like is that a learning around where it breaks down on smaller stuff? I'm just kind of curious.
[00:35:15] Ariel Illouz: I think that, uh, also, you know, uh, in the DevOps world, we decided to exclude those, uh, tasks for now. You know, uh, most of them are small, uh, than three story points as well, but also the big ones. Sometimes, I don't know, in those kind of, uh, stories that more about architecture, like, like, um, infrastructure as code, for instance, that doesn't...
[00:35:42] Ariel Illouz: Is not quite relevant for a spec.
[00:35:45] Andrew Zigler: There
[00:35:45] Ariel Illouz: There
[00:35:45] Andrew Zigler: are some things that are just too big or too encompassing to be frozen into a spec just because of the nature of what they are. That's kinda what you're hinting at.
[00:35:55] Ariel Illouz: Yeah
[00:35:55] Andrew Zigler: and, uh, um, um, another thing I'm curious about too is, like, we talked about the beginning with [00:36:00] the skills, and you have the one skill, the spec skill, the one skill to end them all, right?
[00:36:04] Andrew Zigler: Use that to make everything else. Has your own skill marketplace and practices evolved since then? Do you have more skills? Do people work together on shared skills? What, what does skill process look like at LinearB?
[00:36:16] Ariel Illouz: Yeah, so we have a marketplace. Um, we break it down into external, like, every skill that you would like to, share. Uh, so you put in, like, an external, uh, uh, folder. I think it's a, a different plugin. And we have a internal plugin, which is, uh, skills that, serve the process.
[00:36:38] Ariel Illouz: For instance, the refinement skill that, uh, I just mentioned, that creating, populating the spec document. Also, we are pushing now a second skill, the implement skill. This is something also in motion. And we have many other, uh, more skills. We have the document tool, which, uh, QA is using [00:37:00] in order to get, like, a human-readable document about a feature.
[00:37:05] Ariel Illouz: Uh, we have a test planner skill which populate the tests pl- uh, uh, plan for, uh, any feature out there. We have the troubleshoot skill, which support is using, uh, for troubleshoot, uh, customer support tickets before they are, uh, reaching to engineering. And this is, like, a super success, this skill, the troubleshoot.
[00:37:27] Ariel Illouz: It serves, like, uh, 60% or 70% out of the customer support tickets that are pushing to engineering, uh, stops there by, uh, identifying that this is, like, a settings or con-configuration thing, and getting back with an RCA detailed in the document. Yeah, so this is-
[00:37:52] Andrew Zigler: so it's like these, like you have the, the toolbox, a curated toolbox of these tools that folks turn to. They're, you [00:38:00] obviously have some measured success from them. A key part obviously is having a centralized way of sharing and controlling these skills so they stay up to date. And so I think this is becomes like a really useful playbook for, you know, other engineering leaders that are trying to maybe have the same kind of practice internally.
[00:38:17] Andrew Zigler: We uncovered a lot of what it is, and yours started with interviewing somebody who was successful. Now anyone who's just listened to you for the last, you know, 40 minutes has basically pulled all, everything out of your head about what was successful and what you had to apply. And it comes down to starting with a really small skill, setting a criteria for when you're gonna be using this kind of practice, and then having a shared space where everyone can iterate on the specs together.
[00:38:43] Andrew Zigler: And ultimately, it's resulted in this world where y'all can have a really tight feedback loop. You can ship the software that other teams are using to figure out how they're gonna ship their AI software, so you have to be a little bit ahead of everybody. And speaking of being ahead, Ofer, I'm curious, you know, what [00:39:00] is innovation, AI innovation at LinearB?
[00:39:02] Andrew Zigler: Where's it going next? What do you see the industry evolving towards? We've had some really interesting narratives here, um, in talking about how software development's going to evolve. Spec driven development is just one part of that story. So what do you think about the future of, you know, innovation with AI?
[00:39:21] Ofer Affias: So I think that, um, um, we, we see, the way we see the software industry, it, it's, you know, there's the analogy of, uh, the software factory.
[00:39:34] Andrew Zigler: Yeah
[00:39:34] Ofer Affias: I think some people are like, uh, intimidated by that, but, uh, me coming from a background working with actual factories, with manufacturing, I think it's actually, um, um, uh, like, uh, something to be proud of. There is a lot of knowledge there. Uh, and, but, but specifically I think that, we see it, uh, uh, going that way because the agents are going to [00:40:00] be the, the, the little bots working on, on the stuff And, uh, for evidence, like we all see that coding by humans is something that is going away. Um, um, the, the trends looks like, uh, also testing will, by humans will go away, reviews by guman- humans will go away, and, and so on and so forth. Um, and, and, uh, if we look at, at it with, with this perspective, we look at, um, models becoming better and at what they are doing, like every, three months, six months, uh, we, we, we saw that like, uh, this giant lu- like, uh, jumps in the, in the last year. Um, so it's like we, we'll have... We, we can think about it as l- as loops, and we, we, we talked a lot about this, uh, uh, terminology. So if you think of it, of it like an onion, of like from the inner loops to the outer loops, we are [00:41:00] now dealing with these software factories. The inner loops are being, taken by the agents. As humans, we need to, to adapt and to, to find our value. Um, so agents are taking the coding, they'll take more things, and I think as LinearB, we want to serve that in, in, in... from two ways. One, be able to help with the factory floor. Like, uh, what's now, it was the AI adoption, now it's, uh, the token economy.
[00:41:30] Ofer Affias: We want to make sure that you can optimize your floor.
[00:41:34] Andrew Zigler: Yeah
[00:41:35] Ofer Affias: from the other hand, we, we, as agents will become more and more, like we'll take more and more loops. We want also to make sure that the business is aligned to the factory. we... And, and for that, we want to serve enough context for these agents to be able not only get the build done or the tests or the feature, [00:42:00] but also to make sure that there is impact by the, by the delivery and that it makes sense. Um, and, and this is how we look at it, and this is how, uh, we see the coming. I know, is it, is it good to, uh, to say something about a few years ahead? I think for the next year.
[00:42:19] Andrew Zigler: Maybe. It's like it's so hard to, it's so hard to forecast what it looks like several years ahead. But really what you've hit the nail on here is that when we talk at the factory and we, you know, have this metaphor, we think of the factory, it's an embodiment of these loops that we're talking about.
[00:42:35] Andrew Zigler: The factory is that loop, and what you're describing is having that context available for that loop to be able to be efficient and understand what it's building and what it's shipping. Having the just in time context about your SDLC is what LinearB is building towards, because that is the factory. And the truth is, is that people have a hard time optimizing their factories without having that feedback [00:43:00] loop to help the factory understand itself, improve itself, tie back to the business value.
[00:43:06] Andrew Zigler: So that's what's really exciting about the stuff that y'all are building, and it's been really exciting, you know, Ariel and Ofer, to, to dig inside your heads and figure out how you're building this. So I just wanna thank you both for joining us on Dev Interrupted today. It's been a total blast, and I'm really looking forward to having y'all back in the future.
[00:43:23] Andrew Zigler: Maybe in about, maybe six months we could imagine all these factories everywhere and how we've built, and we could touch back in about this conversation and see where spec driven development has gone next. And to our listeners, you know, if you've been tuning in and listening and curious about the stuff we've been covering today, we've covered a lot of territory.
[00:43:40] Andrew Zigler: We talked about the AI benchmarks that we dropped earlier this year. It was based on, like, over millions of pull requests, uh, for since just January, and you can see a massive trend in how folks are adopting this stuff. So you can start there, and we'll include links to that in our show notes as well. And, Ariel and Ofer and the team are shipping a whole bunch of really [00:44:00] cool, exciting stuff to help teams build and improve these software factories.
[00:44:04] Andrew Zigler: So you can check out more about that on our website, LinearB, as well, which we'll include. Um, and if you enjoyed the episode, give us a, a like, a comment, a review. Come find Ariel and Ofer and I on LinkedIn and poke us, share us your, share your thoughts, disagree with us, and we would love to hear, uh, what you think about your software factory.
[00:44:23] Andrew Zigler: And Ariel, Ofer, thank you again for coming on the show.
[00:44:27] Andrew Zigler: It was a blast to host y'all.
[00:44:28] Ariel Illouz: Thank you.
[00:44:29] Ofer Affias: Thank you, Andrew.
