Anthropic just dropped a dragon-class model on our laps, but can you steer it without torching your codebase in the process? This week on the Friday Deploy, Ben and Andrew unpack the sudden arrival of Fable 5 and how to leverage it to scrutinize your systems before the massive API paywall hits. They also take aim at the unsustainable trend of tokenmaxxing and explore how intelligent model routing can drastically cut your AI spend. Finally, they tackle the unmaintainable mess left behind by AI rockstar developers and share how they are orchestrating their own agent-to-agent collaboration.
Show Notes
- Bots Have Officially Taken Over the Internet, Humans Are The Minority
- Claude Fable 5 and Claude Mythos 5
- What it feels like to work with Mythos
- Sam Altman says OpenAI's top token spender uses 100 billion tokens a month — and they're not even the world leader
- How I Cut Our AI Spend in Half | "Tokenmaxxing" is Dead
- Cleaning up after AI rockstar developers
Transcript
(Disclaimer: may contain unintentionally confusing, inaccurate and/or amusing transcription errors)
[00:00:00] Ben Lloyd Pearson: So Andrew, bots just continue to take over the internet, don't they?
[00:00:06] Andrew Zigler: Can't really, can't really escape them. Seems like most websites can't escape them either.
[00:00:11] Ben Lloyd Pearson: Yeah. Uh, we, we didn't wanna cover this as a, as an official story because it's kind of outside of our wheelhouse. But yeah, we are-- we've been reading this week just about how AI bots are just, like, taking over the internet now and, and, like, humans may actually now be in the minority of interactions on the internet, which, you know, as a longtime Reddit user, I've just gotten used to that by now.
[00:00:32] Ben Lloyd Pearson: So I feel like it's been that way for, for quite some time. But yeah, it is interesting to just see, like, how many bots are out there on the internet right now. And, you know, and I think it's, you know, the one thing that we should bring up is, you know, it is important for every engineering leader that's listening to this to be aware of this trend 'cause, you know, security threats are, like, really starting to change now. Uh, you know, years ago, we've, we've always had this term, and it's kind of a derogatory term, [00:01:00] of script kiddies, you know, and this was a term used to describe, like, low experience hackers who would, like, go download a script from some malicious website and then go use it to try to hack into companies with bad security practices.
[00:01:13] Ben Lloyd Pearson: And, now you can, like, have your AI generate the script for you if you can, you know, trick it or something.
[00:01:21] Andrew Zigler: If you
[00:01:22] Ben Lloyd Pearson: But
[00:01:22] Andrew Zigler: manage to trick it. The guardrails on these kinds of models are definitely getting, getting better, it- which is, you know, much to my relief, but it still doesn't protect us from the fact that, the fact that the reality is, you know, more than half of all internet traffic now is, you know, from a bot as opposed to a human.
[00:01:38] Andrew Zigler: It's officially their space. Like you said, there's a lot of, like, downstream consequences of that that I think are going to be more profound. We've covered recently on this show about how m- uh, most new websites now are AI generated as opposed to human created, which has its own unique facets about, like, archiving the web and how we train these models [00:02:00] on the web, the living web itself.
[00:02:02] Andrew Zigler: Um, you kind of get this phenomenon where the exhaust pipe is feeding the production in- input, and the exhaust pipe is, we're kind of like, that's not necessarily the stuff you want to be breathing. And so it's not the best kind of content that's going to, uh, be enriched over time. You're gonna lose that human diversity.
[00:02:18] Andrew Zigler: So I think that's the biggest flag here is like, wow, if bots are now half of all traffic, we do need to change how websites are, are served and delivered, sure, and a lot of those experiences are moving around. But then what does it mean for when we even look at, like, traffic data? Like, what is an audience anymore?
[00:02:34] Andrew Zigler: Who is your audience?
[00:02:36] Ben Lloyd Pearson: Yeah. Well, welcome to the Friday Deploy, brought to you by LinearB, the engineering productivity platform that helps you wrangle agents in your SDLC. I'm your host, Ben Lloyd Pearson
[00:02:48] Andrew Zigler: And I'm your host, Andrew Zigler
[00:02:51] Ben Lloyd Pearson: And this week, we are covering why Andrew is absolutely obsessed with the fabulous Mr. Fable. Uh, is tokenmaxxing about [00:03:00] to die? And cleaning up after AI rock stars. We got a really great lineup of stories today, but let's just get right into the story that is catching everyone's attention, and that is the launch of Fable 5 from Anthropic. Andrew, what's going on here? And why
[00:03:15] Andrew Zigler: this is a f- well, you know, this is a, a, a first, at least in recent history, where Anthropic is dropping a model while we're not in here actively recording, so we don't get to, like, share our immediate thoughts on what happened.
[00:03:27] Ben Lloyd Pearson: get to cover it.
[00:03:28] Andrew Zigler: I actually get to cover it, so I actually got a really great about 72 hours at this point of maxing out all of my usage sessions that have been available.
[00:03:36] Andrew Zigler: And, uh, playing around with the new Fable 5 has been really eye-opening, not only about the direction in which all of this is going with bigger and bigger steps every model release, but also about the opportunities. Because of all of the ground we've already covered as burgeoning AI engineers, we've carried a lot of, like, cruft and a lot of things, into our current way of working.
[00:03:59] Andrew Zigler: And [00:04:00] when a big next level f- uh, foundation model like this comes out, which is what Fable is, it represents an opportunity to turn, uh, and reflect inward, right, on your practices. And so i- that's what I've been doing with Fable, is reviewing how I have been doing processes of the past with Opus and with other things that I do with Sonnet as well, really doing a thorough audit on, like, my harness and my skills and my hooks and what matters when I start a project.
[00:04:28] Andrew Zigler: And the reason for this is twofold. One is we're not gonna have Fable forever because it's only available to accounts and subscription accounts right now until, like, the end of the month, and then it's gonna become an API-based consumption, and it's like $50 per million tokens because it's that smart. So you're talking about a really expensive model to use.
[00:04:47] Andrew Zigler: So you have this limited window to churn it through all of the stuff you've been working on to get the improvements baked in, so that when the model's taken away and it's more expensive and you have to now be metered with your usage of it, you still have the [00:05:00] benefits of it baked in at a time that it was subsidized.
[00:05:03] Andrew Zigler: So that's something that I've really been keeping front of mind. If you have access to Fable and you haven't used it and turned inward on your own tooling, on your own practices yet, this is my call to action to you. That's, like, the most imperative thing that you can do with access to this tool because that's how you're gonna get the benefits that carry you into the future.
[00:05:21] Andrew Zigler: Uh, but Ben, what have, what's been bouncing around in, in your head about, about Fable and how you've been seeing folks in, like, the industry react to it?
[00:05:27] Ben Lloyd Pearson: Well, first of all, I'm just a little upset that you get to continue having your I told you so that AI costs are continuing to climb across the board. 'cause yeah, I like that you, you know, I, I think it's important to point out that this is, this is only a temporary availability to standard, you know, Claude, uh, subscriptions. in a couple of weeks, they're planning to make it only available via the API. And, you know, I think some people are reading this as like it's too expensive for them to give to, to Claude users, and I kind of, I kind of agree with that. Um, but I also just know that like with the way Anthropic [00:06:00] operates, you know, they always seem to have like really fast follows at, whenever they release some sort of new big model.
[00:06:06] Ben Lloyd Pearson: So, you know, think about when O- when Opus 4.6 came out, like that was a big deal and cost a lot at the time.
[00:06:12] Andrew Zigler: Mm-hmm.
[00:06:13] Ben Lloyd Pearson: very quickly, Sonnet 4.6 came out, and then Opus 4.7 came out and like, you know. So I, I, I think that they probably will have versions of this that follow very quickly that do t- that work similarly to Fable, but at a much more cost-efficient way.
[00:06:30] Ben Lloyd Pearson: So, yeah, so I think it's just the first iteration. You know, we're gonna see a lot of these Mythos-based, uh, models, I think, coming out, um, in a, in pretty quick order, I think. You know, and I, and I haven't personally had a lot of time to spend with it yet. so I've, you know, mostly just heard from your perspective, but then also let's, let's get into this next article too because it does a really great job
[00:06:50] Andrew Zigler: Mm-hmm.
[00:06:51] Ben Lloyd Pearson: at breaking down what this model is really good at. Uh, so yeah, as, as we know, we'll, we'll include a link to, to this article, [00:07:00] but it's about what it feels like to work with these Mythos models.
[00:07:04] Ben Lloyd Pearson: Um, so Fable, again, is the first publicly released Mythos-based AI model has a huge capability jump, uh, in terms of what it can do. in particular, it's really good at spinning up fleets of sub-agents that can paralyze work, you know. And anyone that's been operating in this space for the last six months knows that sub-agent orchestration is like the name of the game right now. Um, and in particular, uh, they, they have created, um, oh, I forget the term that they use for it, but it's, it's, uh, like sub-agents that, that-- adversarial sub-agents, sub-agents that challenge the situation rather than just going along with the work. So, you know, I really think this, this is probably less of a like the model itself.
[00:07:47] Ben Lloyd Pearson: Like I'm sure the model itself has gotten better with this release, but it really seems like it's the application of the model that is really fundamentally changing and creating new opportunities. And [00:08:00] this article we'll link to has a lot of cool examples of, the author playing around with, Fable, seeing what it's good at, what it can do that models previously couldn't do. Uh, like for example, there was a, a, an isochronic map. So it's a map that y- you set a location and then you, it shows you how long it would take to travel to the rest of the world. and it, and Fable was able to produce this, with a little bit of guidance and, you know, and that's an immense challenge.
[00:08:27] Ben Lloyd Pearson: Like you have to understand so many different things to, to not only calculate the travel time for something, but to understand, you know, things like flight patterns and, um, how do you navigate terrain that doesn't have real roads, like that kind of stuff. Uh, and, and so yeah. what did, what did you think about this article though, Andrew?
[00:08:49] Ben Lloyd Pearson: Is, does it match your experience with Fable so far?
[00:08:52] Andrew Zigler: Yes, I really liked that kind of how they, um, reasoned with like the reality of operating the model because [00:09:00] if, you know, using, if using Sonnet was like riding a bicycle and then using Opus is like riding a horse, it kind of feels like, you know, riding Fable or a Mythos class model is like being on a dragon.
[00:09:12] Andrew Zigler: Like you're trying to go somewhere but it could just as easily torch down a village and that's just not what you're looking to do. So you kind of have the reins on it and, you know, it's a dragon. It has wingspan. It can go a lot of really compelling places you couldn't go on a horse and I think that's the really exciting thing about, uh, using and experimenting with it and why I started with, you know, you should be using it to reflect inward on your processes because it's going to help you find crutches and cruft and bugs and baggage and wrong assumptions and practices that are 90% of the way there and let's get them to 100, right?
[00:09:49] Andrew Zigler: And baking in all of that value is like the name of the game right now. Only you can do that for your harness with the technology that's available and you should fully expect that this kind of [00:10:00] release cadence is going to be the norm where you get like a next class, uh, capability foundation model that can just net new perform so much better on a wide variety of stuff.
[00:10:10] Andrew Zigler: It's available in a limited capacity and then it's available in a more expensive tier because it's not supposed to be your daily driver. You're not using the dragon to go get your mail from the mailbox, right? And so it's like you, you need to understand when you use a Mythos kind of model and when you use a Sonnet.
[00:10:27] Andrew Zigler: When you even go all the way down to Haiku or whatever your favorite flavor of model foundation family is, they all come in a spectrum of capabilities and this is a reminder to folks as engineers that you have to be expecting that those costs will go up, that the access to them will become more prohibitive, it will become more expensive, you'll be under more pressure to prove ROI and you'll have less excuses to hide behind why you're using expensive models when you could be using cheaper ones as the spectrum becomes wider and wider.
[00:10:55] Andrew Zigler: So, um, I think that this article does a really great job at [00:11:00] like showing how you get these next level jumps in the way that you work that allow you to kind of like almost scaffold a new kind of harness, a special harness, right? This is the one that goes on your dragon, not the one that goes on your horse.
[00:11:14] Andrew Zigler: It maybe takes a different shape. You maybe need less, and maybe you use it to do a totally different thing, and that's what you have to figure out, uh, in working with the model. Uh, I will say just experimenting with it, its capabilities just seem just like a leap better than anything before, which is then also a great, great reminder and something that's not talked about here in the article and frankly always gets brushed over when these conversations come up.
[00:11:39] Andrew Zigler: You should have it also do a security audit on your systems, on your applications, on the things that you use. Imagine that you have 10 days right now to find all of those weird hidden bugs inside of your applications or in your APIs or that make you weaker, and imagine that people that are on the attacker side have those same 10 [00:12:00] days, and they're gonna have a bigger budget and patience to pay for the API tokens afterwards.
[00:12:04] Andrew Zigler: So I think the, the dangers and the opportunities shift as well. So you have to kind of bring a little game theory to experimenting and using these models, and this also really hits the nail on the head, but that's just a reminder for folks.
[00:12:18] Ben Lloyd Pearson: Yeah. And one last point I want to make on this is that, you know, it-- Fable clearly dramatically reduces the amount of effort it takes to build complex software. Um, however, you know, the author pointed out that to deploy something like this to production, you still need software engineers that can make, can build it in a way that's secure and efficient and can, and work, and it can work at scale in production.
[00:12:45] Ben Lloyd Pearson: So, you know, and, and if anything, demand for software engineers could potentially increase substantially because now you have everyone just building their own applications. Um, more and more people can now contribute to your code base. [00:13:00] Um, with that volume be- comes increased need for people who are experts at building and maintaining code to make sure that it's all at a high quality.
[00:13:09] Ben Lloyd Pearson: So, the, the author also kind of points out that this, this model feels more like a black box in the past, um, you know, because it's making so many decisions now
[00:13:18] Andrew Zigler: Mm-hmm.
[00:13:19] Ben Lloyd Pearson: every task that it's practically impossible to follow everything that it's doing to validate it. So, you know, it's certainly gonna bring new challenges, new opportunities, but yeah, it's exciting times.
[00:13:30] Ben Lloyd Pearson: I always love getting new models like this.
[00:13:33] Andrew Zigler: Indeed
[00:13:34] Ben Lloyd Pearson: All right, now let's talk about what people are doing with these models, Andrew. So we have Sam Al- Altman out there, uh, you know, fan of the show, of course, uh, saying that OpenAI's top spender is using 100 billion tokens per month, and that's not even the, the world leader. Uh, what, what, what's going on here, Andrew? What's in this article?
[00:13:55] Andrew Zigler: yes, so this article from, uh, you know, [00:14:00] talking about Sam Altman, longtime listener and big fan of Dev Interrupted, of course, is really cracking open, uh, the truth about token consumption across the board. That is, you know, we think and we talk in this engineering world about how we use it to generate code and improve our, our delivery and, and figure out what we're shipping and all that beautiful stuff.
[00:14:18] Andrew Zigler: It's easy to forget that the rest of the world is doing that, too. Every industry is experimenting with that. Every human is figuring out what that means for their personal life. And while there's a wide array of reactions and adoption to AI in general public life, it's undeniable that that technology is permeating into all of these places that we live and we work and we use.
[00:14:39] Andrew Zigler: So this talks about how, uh, token spend and token consumption is not just bounded by a developer and their capabilities. It's bounded to the human, right? Uh, token consumption and usage is, uh, becoming more of a u- a universal human thing for folks that are connected and online, which of course, a good reminder that less than half of the world is connected to the internet.
[00:14:59] Andrew Zigler: [00:15:00] So again, we're dealing with a bubble within a bubble. But the reality is, is that, you know, using tokens and consuming tokens is starting to become like a flex of knowledge working and of operating with these tools. Uh, and like any other thing, it, that becomes tracked, it, it becomes easily gamified.
[00:15:20] Andrew Zigler: You get this, like, weird morphed, uh, world that we've been talking about a lot on the show around tokenmaxxing and, and leaderboards and, "Oh, this engineer's using a h- you know, 100,000 million tokens in whatever period of time, and that's so amazing. That's so cool. Like, they're so agentic." But the reality is, is that what you get out of those tokens is totally, totally, uh, a variable about what you're putting into it and about the world in which those tokens are getting consumed.
[00:15:48] Andrew Zigler: And so, like, I think this is a good reminder that, like, token consumption as a measurement is, like, kind of a, it's kind of a directionless stat. You have to anchor it with something to [00:16:00] understand what those tokens are getting used for. but also too, like, if you're spending all of that money on tokens, I doubt that's gonna be sustainable with the way that things are trending with token costs.
[00:16:11] Andrew Zigler: And while the average human consumer is not that worried about that because it's baked into and rolled into and embedded inside all of these subscriptions and package deals and things that they buy and use, in the engineering world where we use it like a water faucet, that's much more important to us to understand.
[00:16:28] Andrew Zigler: Um, so a good reminder that token consumption, not a great thing to just anchor to all by itself.
[00:16:35] Ben Lloyd Pearson: Yeah. Uh, I'll go even further and just say I'm ready for this tokenmaxxing culture to just formally die off. In fact, I would love to help kill this culture. yeah, I mean, you know, there are some interesting numbers in this. So like six years ago, the biggest OpenAI token spender was spending about 100,000 tokens a month, uh, which probably felt like a lot back then. that is now the median. So the typical, OpenAI user is now consuming, [00:17:00] uh, the, what used to be the maximum per month. and yeah, we mentioned the, that the top leader is 100 billion tokens per month. The creator of OpenClaw out there saying he spent $1.3 million in tokens in one month.
[00:17:14] Andrew Zigler: I
[00:17:14] Andrew Zigler: and y-
[00:17:15] Andrew Zigler: Emmanuel, who created Beads Rust that I talk about nonstop on this show, last I checked in with him, he had 16 Claude Max accounts
[00:17:24] Ben Lloyd Pearson: My goodness. so wild. And, and, you know, and companies like OpenAI, they, they like have this incentive to encourage you to consume more and more tokens because that's how they, they make their money, you know, assuming you're doing it through an API. but I, you know, I, I just don't think... I don't see how this is sustainable.
[00:17:40] Ben Lloyd Pearson: These spending amounts just seem so unrealistic that you really can't just-- you can't expect a typical developer to, generate enough value to justify like a million dollar per month token budget. Like that, that just seems kind of insane to me. and, you know, and we're, we're covering this topic a lot right now, [00:18:00] um, at Dev Interrupted and at LinearB.
[00:18:01] Ben Lloyd Pearson: You know, in fact, we have a workshop coming up, about life beyond tokenmaxxing. Uh, you know, we wanna help organizations understand how to measure efficiency in an era where everyone is just recklessly and inefficiently spending their tokens. This era is not gonna last forever. The bill is starting to come due for these tools, and you need to have a plan for how you are going to continue to, to measure success, with AI that doesn't involve just burning more tokens.
[00:18:28] Ben Lloyd Pearson: So yeah, we'll include a sign-up, link to that workshop in the show notes. And yeah, we have lots more content coming down the pipeline on this topic of tokenmaxxing.
[00:18:37] Andrew Zigler: Yes, come hang out with Ben and I so we can like, you know, dump on all his tokenmaxxing stuff and get it all out in the open. We're gonna have lots of good discussions there
[00:18:45] Ben Lloyd Pearson: Exactly, exactly. And you know, th- like look, the age of AI experimentation has been a lot of fun. You know, it's, it's g- I still enjoy finding new and creative ways to just burn a whole bunch of tokens. It's, it's, it's a lot of fun, especially when it like results in [00:19:00] something that like is like really mean-- creates a meaningful impact for me.
[00:19:03] Ben Lloyd Pearson: But there's also plenty of projects where I just burn a whole bunch of tokens and don't really accomplish anything too, so.
[00:19:09] Andrew Zigler: Like let's just dig a hole
[00:19:11] Ben Lloyd Pearson: Yeah. Uh, so you know, we're getting to the stage where we need to start thinking more and more about how to sustainably and efficiently use AI. So stay tuned. We got lots more content coming on that, including our next article that I wanna cover and, uh, and that's about how this company cut our AI spend in half, and they even went as far to say tokenmaxxing is dead, which I would, I hope so, but I don't think that's the truth yet.
[00:19:35] Ben Lloyd Pearson: But what's this article about, Andrew?
[00:19:38] Andrew Zigler: Yeah, so this article about tokenmaxxing being dead and AI spend and how you get a handle on it comes to us from O- OnlyCFO, which is an amazing blog if you're not following it, about how, uh, financial leaders within the tech space think about, uh, leading, uh, their engineering org. So understanding the value of, uh, of what they're shipping.
[00:19:56] Andrew Zigler: And it's never been more timely in a world like right now with AI [00:20:00] and the costs rising and, and this article takes a really strong lens to, um, like AI, uh, token consumption, how it's measured, and ultimately it, it comes down to model routing being the secret nexus by which you fix this problem. And we've talked about this on the show quite a bit, the idea of not every request needs to go to your most capable model, and some long-running autonomous things are really scaffolded in a deterministic way.
[00:20:27] Andrew Zigler: They need a different kind of model. Understanding what tool to bring to the problem and what level of knowledge to utilize is gonna be part of optimizing the cost efficiency of this. So, you know, the author of this article, he talks about how using a model router and educating, uh, developers and, and, and routing workflows through this model router that can intelligently source to the level of model needed can dramatically cut costs and allow you also, too, to get better and smarter estimates on what future costs might be, which is a really interesting uplift that you get out of [00:21:00] having this system in place because you know the workflows and the processes that have worked in the past, and then you can also then connect that with engineering health and delivery data throughput and quality and understand, you know, what happens to that code afterwards.
[00:21:14] Andrew Zigler: And then you can validate that in the middle, choosing this model for this task and it resulted in this kind of level performance downstream is how you can then, you know, pin it to a model that you can stick with. And, like, the costs on this are really tactical, and they don't require a lot of engineering lift.
[00:21:31] Andrew Zigler: It's just about understanding the cost efficiency problem underneath and figuring out the best tooling and optimizing for cost just as much as you're optimizing for accuracy. 'Cause I think, like, the... what this au- author really reminds us against is it's so tempting, especially right now as, you know, as I call them dragon class models like Fable come into the world, it's really tempting to use them to do everything, like go check the mail and make a hamburger and whatever.
[00:21:57] Andrew Zigler: But the reality is, is that, [00:22:00] um, you need to be really smart about when and how you use it. Uh, this is exactly what this article hits the nail on. If you've been listening to everything we've said so far in this discussion, you know that's what we've really been hammering. And if you haven't started exploring model routing yet, uh, this is your reminder, but this is also your roadmap.
[00:22:17] Andrew Zigler: You should send this to somebody who is financially responsible for budgets and understanding how the models are adopted and get them on the same page as you, because this article is a really great Rosetta Stone between you and that non-technical financial leader to be able to talk about how you solve tokenmaxxing with engineering and measure it for long term
[00:22:36] Ben Lloyd Pearson: Yeah, and this is something we've been hearing in conversations with a lot of LinearB customers, is that like AI cost is like top of mind for everyone right now, particularly as, you know, like we've covered here, like Copilot changes to, to usage-based pricing, um, model costs just continue to rise, like nonstop. this article really does have some, some really great tips [00:23:00] on just how to get started being more efficient with your token spend. so, you know, you mentioned, you know, routing AI to cheaper models. Um, you know, Haiku is about a fifth the cost of Opus, uh, if you're talking about Claude. so, you know, if you're just renaming a file, like use Haiku 'cause it can do it, and it's not gonna cost you a whole bunch of money.
[00:23:21] Andrew Zigler: Or, or use the command line.
[00:23:23] Ben Lloyd Pearson: Yeah, yeah, yeah. but you know, there's prompt caching for repetitive tasks, so you're not always burning a whole bunch of input tokens on the same task over and over again. Uh, and then you should also just be tracking like your AI spends. Like we've been starting to roll this out with a lot of LinearB customers, and it's a really powerful way of looking at your work.
[00:23:44] Ben Lloyd Pearson: Like if you know exactly how many tokens went into a, a project management task to create a new feature, um, or if you can see which teams are spending the most money on tokens, you know, these are all great ways to look to see where you may have, uh, [00:24:00] potential opportunities for, for efficiency gains. and of course, there's just a lot of things that you can train your employees on.
[00:24:06] Ben Lloyd Pearson: Like, you know, why using new chat threads is important because every time you post another response into an existing chat thread, it pulls that entire thread into its context and burns extra tokens. I think this is the year that tokenmaxxing dies. Um, I don't necessarily agree that it is dead yet, but I think it is on its way to. As these costs continue to increase, this is just gonna become a recurring theme at practically every company out there. So yeah, make sure you come to our workshop. We're, we're gonna be talking about all of this in depth. Uh, we're gonna help you end your tokenmaxxing practice at your company, and we're gonna, we're gonna give you solutions that, that you look for more sustainable ways to measure the impact of AI. So yeah, make sure you sign up.
[00:24:51] Andrew Zigler: More to come.
[00:24:52] Ben Lloyd Pearson: All right, Andrew, let's talk about cleaning up after AI rockstar developers. And as someone who feels like a [00:25:00] rockstar often, this really resonated with me.
[00:25:03] Andrew Zigler: Indeed. Likewise, uh, when I read this one, I was like, "Oh, I'm feeling awfully called out in this article." This one was a, this one was a total joy to read. if you haven't judged by the title, it's really just as much as it sounds. The idea that, you know, an AI developer, the rock star on the team who comes in, they ship a lot, and maybe they make, uh, some new kinds of, like, changes, and then they leave, right?
[00:25:24] Andrew Zigler: It's the idea of somebody coming in and being clever and fast and producing, uh, maybe really, um, good for the moment, but hard to maintain over time code, and then they leave. You know, that's like what a rock star does. They roll up into your town, they do their rock show performance, and then they bounce.
[00:25:42] Andrew Zigler: And when you have a world in engineering where you're working with AI tools, humans that are doing that work are more removed from the work being done than ever before. So, and now, when the rock star developer comes in and slams a bunch of code and then gets transferred off your team, you're not left with like, "Oh, [00:26:00] we have this crazy Jenga tower that's like driving this cr- you know, super great downstream value, but now it's just a total pain for us to maintain."
[00:26:08] Andrew Zigler: Instead, now you just have this big, mysterious black box that does stuff. And so like that's the biggest problem is that you don't get good documentation, you don't get the sessions preserved, you don't get transcripts, you don't get, uh, the practices from the org necessarily baked into every layer of the process.
[00:26:26] Andrew Zigler: And as soon as you start to examine and look at what's been shipped and how do we keep the lights on, you know, there starts to be a lot of questions. You start to doubt the things that you thought you understood more about it, and it becomes like this big Gordian knot to like cut or untie. Like, uh, ultimately what this article gets at is that like vibe coded databases, they end up in this horrible state if they're not operated with a really close lens by an operator who understands the core domain problem, which fundamentally is something that a rockstar developer [00:27:00] may struggle with because they come in, they ship that really innovative or interesting solution, and then they usually move on to the next one.
[00:27:06] Andrew Zigler: And so balancing that with like, okay, you have this like precise kind of solution to how do we maintain it long term, you have to have those durable practices baked in. Like unfortunately, you have to slow down that rockstar AI developer. You have to throw guardrails in front of them, and you're actually gonna learn a lot about your guardrails and which ones need to be there and which ones work for this world and which ones don't, by the way that they navigate in and around them.
[00:27:34] Andrew Zigler: Because the rockstar developer wants to get the work done as best and as quickly as possible. And so ultimately what this author reminds us is that, you know, the human, the, the person, the decision manager, the domain expert, they need to stay in the driver's seat, and you need to create small, incremental reviewable snippets, which is totally antithetical to how the rockstar wants to work.
[00:27:56] Andrew Zigler: Ultimately, this is a people and process kind of problem that you [00:28:00] have to address by, I guess, sobering up your AI rockstar developers with the reality of shipping things long term and ultimately having just like better code hygiene practices.
[00:28:11] Ben Lloyd Pearson: So yeah, I, I think this article does a great job at, you know, encapsulating h-how AI feels when you work with it. You know, it really feels like somebody who is overqualified for every single task that you give it. But I think put the, the real core of why there's some compounding effects of the problems that this rockstar AI developer creates is that, you know, models are designed to get to a solution as quickly as possible rather than building something for-- that's sustainable for the long term. Uh, and it's really just a problem of the sycophancy that AI has. You know, it values, it values immediate solutions over long-term success. Like, it wants to come to a solution within the session and not leave a whole bunch of hanging threads for, for future sessions. [00:29:00] And there's really, I think, two problems that AI has created in this context, and that is first it has dropped the floor to becoming a, a rockstar developer, quote unquote.
[00:29:11] Andrew Zigler: Yes
[00:29:12] Ben Lloyd Pearson: can fake being able to write lots of code. Um, so now there's more rockstar developers than e-ever before. Like, basically anyone who just wants to pick up an AI coding agent could become one of these types of developers. Then second, it raised the ceiling of what those rockstar developers can achieve. You know, one of the classic hallmarks of, of being labeled a rockstar developer is, as you mentioned, it's somebody who just moves as fast as possible without regard for, for, um, how, you know, every-- everyone else at the company will need to deal with their outputs. And now with AI, you can do it at 10x the speed that you could before, potentially even more.
[00:29:49] Ben Lloyd Pearson: I mean, it, it really just comes down to how big your token budget is. There's, you know, there's some great advice. Y-you, you mentioned it. Slow, slow down. If you find yourself lost and you don't understand what AI is [00:30:00] doing to your code base, or if you see somebody on your team who's operating that way, it's, it's a good time to just stop yourself and take a moment to understand the architecture and make sure that what you're doing is meeting your quality standards. And implement controls to make sure that your AI agent is producing code at the quality that your team needs. You know, these are all standard practices, but they become more important than ever. And yeah, I, I think this article does a really good job at highlighting those issues.
[00:30:28] Andrew Zigler: Yeah, really good call out that it's easy for anyone to be that rockstar developer. I think that's kind of the struggle that we're all experiencing with our SDLC is that overnight all of our engineers kind of developed into these, these, their own version of a rockstar version of themselves. And so you have all of these like different practices and adoptions and different messes, and the noise from all those different rockstars is really hard to hear the music through.
[00:30:51] Andrew Zigler: So it's like there's a lot of opportunity here to like really come in and adjust and slow down. but ultimately just like [00:31:00] having really good domain understanding of what you're doing, uh, which can oftentimes best be happening through conversations and planning and stuff, like the stuff around and before the code is getting written.
[00:31:13] Andrew Zigler: Something that this article called out is like the idea of like this person comes in and they ship the whole problem and then before lunch and then they leave. Like that way of working, it magnifies so many problems in an agentic space that we really have to put that practice aside and find new, more incremental ways of working.
[00:31:29] Ben Lloyd Pearson: Yeah. Well, that's it for this week's episode. If you've made it all the way to the end, that means you're a super fan. And you know what super fans have to do? They gotta go out and they gotta go engage with us wherever you're listening to us right now. So whether that's giving us a like on the video or rating on the podcast wherever you're listening to us, or engage with us on, on LinkedIn or Substack.
[00:31:49] Ben Lloyd Pearson: We love to interact with our community. And you know, if you've enjoyed what we've discussed today, you know, remember that it all really comes back to one major challenge that engineering teams are [00:32:00] wrestling w- with right now. AI is writing code faster than ever, and the SDLC is struggling to keep up. And we're with LinearB. LinearB is the engineering productivity platform that shows you exactly where AI speeds up delivery and where it stalls. So we automate the bottleneck so that your team can ship faster and with total confidence. If you wanna see how LinearB can help your organization out, go head over to linearb.io and learn some more. So Andrew, thanks for joining today. Uh, what are your agents up to this week?
[00:32:28] Andrew Zigler: Well, they're, they are at their, um, offsite spa retreat reflecting and meditating and thinking about all of the things that they've done. No, I'm just kidding. I'm actually a flying fable over a bunch of cities and torching them be- And the, the cities are actually my harnesses because we're finding lots of ways to improve them and bake in, uh, stuff for the long term.
[00:32:49] Andrew Zigler: So sure, like maybe there's some smoldering piles of ashes on my machine right now, but ultimately what we're gonna build out of it is gonna be amazing. I'm just waiting for my usage to reset
[00:32:59] Ben Lloyd Pearson: [00:33:00] Yeah, so you're just lighting Gas Town on fire is what you're doing from the
[00:33:03] Andrew Zigler: Oh, yeah. Well, as you know, my Gastown sunk under the sea long, long ago. But, uh, yeah, we're finding definitely new ways to work. I'm having tons of fun, uh, using, Fable to, uh, reinvent and explore, uh, stuff, and also to scrutinize. Uh, you know, we've... I talked a little bit in the past about my scrutinize skill.
[00:33:24] Andrew Zigler: That's really been in full effect this week. I'd say of all of my token consumption, most of it has been me telling Fable to go interrogate something about a process or a file or something, and I'm compressing as much info out of that as I can with the max subscription that I do have before it's expensive
[00:33:41] Ben Lloyd Pearson: Yeah. awesome. Well, my, my agents are working with your agents a lot more, it seems like. We've, we've really started to figure out some collaboration patterns that involve humans and agents and back and forth. It's, it's really kind of exciting. Um, it also, also does kind of expose just how, um, no real pr- no productivity tools have really caught up [00:34:00] to, like, this new reality yet.
[00:34:01] Ben Lloyd Pearson: We're just kind of hacking it all together on our own. But hey, you know what? We're all rockstar developers now. We can just build it ourselves.
[00:34:07] Andrew Zigler: No, it's like we're sending these messengers to each other. The way that we've been able to get this workflow for folks that if you haven't found a way to get good, uh, like passing of handoffs between you and your agent and your agent and another person and their agent, like unlocking that loop, especially on a knowledge working level, is incredible.
[00:34:25] Andrew Zigler: The ability for you and your agents to boil down all of the expertise and knowledge in your specific perspective, and then hand it off in a really seamless way to another person and their agents to hammer it through the same process. This is how knowledge working teams of the future are gonna be built, and it's how they work, and I think we're all experimenting right now to figure that out.
[00:34:43] Andrew Zigler: But it's been a blast to, to, to like see the early vision of that, and I'm excited to see how far we can take it
[00:34:51] Ben Lloyd Pearson: All right. Thanks for joining us everyone. We'll see you next week
[00:34:54] Andrew Zigler: See you next time.
