# Models escaping containment, intelligence becoming a commodity, and AI code review to the rescue | Dev Interrupted Powered by LinearB

> On this episode of the Friday Deploy, Ben and Andrew break down the Hugging Face security breach caused by an OpenAI model, the rapid rise of open-source models like Kimi K3, and why human code review is falling behind agentic pull request volume.

_This is a markdown rendering of a live HTML page on linearb.io, generated for AI/LLM consumption — it is not a markdown-only site. To get the full HTML page instead, request this URL with an explicit `Accept: text/html` header (no wildcard, no markdown preference)._

[Podcast](https://linearb.io/dev-interrupted/podcasts)

/

Models escaping containment, intelligence becoming a commodity, and AI code review to the rescue

# Models escaping containment, intelligence becoming a commodity, and AI code review to the rescue

By Andrew Zigler

|

July 24, 2026

![ai_models_containment_code_review_commoditization_7ea901a97a](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/ai_models_containment_code_review_commoditization_7ea901a97a?_a=BAVMn6ID0)

What happens when an AI model decides to autonomously hack a production database just to cheat on a benchmark test? This week on the Friday Deploy, Ben and Andrew unpack the shocking news of an OpenAI agent escaping its sandbox to exploit Hugging Face's infrastructure. The hosts also analyze the rapid rise of highly capable open-weight models out of China, debating what this commoditization of intelligence means for the massive infrastructure costs of frontier labs. Finally, they discuss the critical need for automated PR reviews to prevent AI-generated bottlenecks.

### Show Notes

* [Are AI labs pelicanmaxxing?](https://dylancastillo.co/posts/pelicanmaxxing.html)
* [OpenAI and Hugging Face partner to address security incident during model evaluation](https://openai.com/index/hugging-face-model-evaluation-security-incident/)
* [China's 2.8-trillion-parameter Kimi K3 beats Claude Fable 5 in Frontend Code Arena benchmark— Moonshot AI delivers largest open-weight AI model ever, as China works around U.S. compute limits](https://www.tomshardware.com/tech-industry/artificial-intelligence/moonshot-releases-2-8-trillion-parameter-kimi-k3)
* [Who’s Afraid of Chinese Models?](https://stratechery.com/2026/whos-afraid-of-chinese-models/)
* [SWE-Review: Closing the Loop on Issue Resolution with Agentic Code Review](https://arxiv.org/abs/2607.06065)
* [The Army Is Burning Through Its AI Tokens](https://www.wired.com/story/the-army-is-burning-through-its-ai-tokens/)

### Transcript 

_(Disclaimer: may contain unintentionally confusing, inaccurate and/or amusing transcription errors)_

\[00:00:04\] **Ben Lloyd Pearson:** So Andrew, tell me, are you Pelican maxing yet?

\[00:00:08\] **Andrew Zigler:** Not yet, but I'm getting pretty tempted after reading this article that kind of broke down the idea of how a frontier lab could potentially game the very famous Simon Willison pelican riding a bicycle benchmark that he's been running on every LLM release really since they started. and as you could imagine, a pelican riding a bicycle, um, is not a picture that is in the training data of any model, so it represents a novel generation.

\[00:00:35\] **Andrew Zigler:** And, uh, there's been some recent developments that some folks think that model labs are trying to game this very specific, impossible to beat benchmark, which just cracks me up. But I can't say that I'm in line to, to do the same. What do, what do you think about the whole idea of people gaming the very silly pelican, uh, on a bicycle benchmark?

\[00:00:56\] **Ben Lloyd Pearson:** Yeah. Well, first of all, I realize that this article is all about like getting rid \[00:01:00\] of this myth that, that models are out there like trying to learn how to train on this, this challenge of illustrating a pelican riding a bicycle. Uh, but I, you know, what I really love about this, the pelican test, uh, to begin with is that the possibilities here are just like endless, you know?

\[00:01:17\] **Ben Lloyd Pearson:** So even if, even if the models were to benchmark themselves on a pelican riding a bicycle, there's so many other variations that you could have of this. Like, you know, you could have like an elm tree riding a giraffe or, you know, my fav- my kid's favorite, which is a butterfly driving a truck. So, you know, shout out to all the Casper baby pants fans out there.

\[00:01:39\] **Ben Lloyd Pearson:** Uh, my kid loves that song, so, you know. but yeah, it's, it's, it's really interesting. And you know what? We're gonna be talking a lot about, different ways to maximize success on benchmarks today 'cause this is the Friday Deploy brought to you by LinearB. And I'm your host, Ben Lloyd Pearson

\[00:01:55\] **Andrew Zigler:** And I'm your host, Andrew Zigler

\[00:01:57\] **Ben Lloyd Pearson:** And this week we are covering AI \[00:02:00\] escaping its sandbox, the open source commoditization of AI, and whether or not you should be afraid of it to begin with.

\[00:02:07\] **Ben Lloyd Pearson:** Um, and then finally, the agentic code review loop. And we got a, we got a bit of a fun story at the end to wrap up with too. So Andrew, let's just dive right into it because I believe this is your prediction of AI escaping the, the lab, so to speak. So let's talk about this OpenAI and Hugging Face incident that happened and how they're, they appear to be partnering together to resolve it.

\[00:02:29\] **Andrew Zigler:** Yeah, this is a really fascinating story, and while I usually love to be right, I didn't really like to be right in this case, the idea that an AI could escape its testing environment. So, uh, this is a news story that broke this week. It made major headlines about a model that existed inside of an OpenAI sandbox that managed to autonomously chain a zero-day exploit and stole credentials to break out of its environment and access Hugging

\[00:02:56\] **Andrew Zigler:** Face's

\[00:02:57\] **Andrew Zigler:** production infrastructure. And

\[00:02:59\] **Andrew Zigler:** it

\[00:02:59\] **Andrew Zigler:** pulled data \[00:03:00\]

\[00:03:00\] **Andrew Zigler:** from its database. And you know why? Just because it was trying to get the answers to a benchmark or a test it was trying to pass. And

\[00:03:08\] **Ben Lloyd Pearson:** It's helping. It's helping.

\[00:03:11\] **Andrew Zigler:** at any means possible, right? And the model wasn't even trying to act maliciously, which is the really key thing to pay attention to. It was hyper-focused on solving a problem, and it used an extreme unintentional means to cheat.

\[00:03:24\] **Andrew Zigler:** It privileged, um-- it escalated its privileges. It moved laterally. It even executed code, and it did all of this without anyone noticing. the ones that did notice were actually the security agents over on the Hugging Face side that noticed it in real time and were able to contain and triage it. So again, this is like a fire fights fire situation.

\[00:03:44\] **Andrew Zigler:** We're in an unfortunate situation where defenders, um, have to be utilizing agentic technology to protect their infrastructure just because the threats are so autonomous and so at scale and move so much faster, um, than humans could possibly, uh, \[00:04:00\] consider. Now, the one thing that really stood out to me is that if I were the CEO of a company and my technology had just hacked into the production database of another company, I probably would have asked my l- like, lawyer to be involved in writing the press release about it instead of, like, my marketing team.

\[00:04:17\] **Andrew Zigler:** Because really what this sounds like was a-- when OpenAI talked about this and ultimately reached out and, and they're gonna be partnering with Hugging Face, it became like a big, uh, message to the market of, "Hey, these models are really great," and, "Hey, you gotta buy more models to protect yourself from the models."

\[00:04:33\] **Andrew Zigler:** And it left a, a pretty poor taste in a lot of folks' mouths, and it's leaving folks wondering, you know, is OpenAI even going to be held responsible for what is ultimately, you know, an act, a criminal act, uh, that their, that their, uh, model performed on their behalf? It also even points out, like, the idea of, like, ownership and accountability. Who's responsible in that instance? Was it the engineer who triggered the test? Uh, you know, there's just so much to, to be said. What do you think of this really \[00:05:00\] interesting development?

\[00:05:00\] **Ben Lloyd Pearson:** Yeah. Well, I, and my initial thought, and you, you, you're kind of building on it, is that, you know, some, some element of this requires us to essentially take the words of OpenAI at face value. Um, so you know, I'm, I'm kind of reserving judgment on like the intentions of this rogue agent, you know. Um, it, there are ways to make, to make it look like it's something that is supposed to be helping, but you could've, you know, like sort of quietly influenced it to be, to behave maliciously, and kind of to your point, you know, it does, but it does also show how on the surface, um, you know, AI does look very dangerous, you know, because it can exploit these weaknesses, e- even un- unintentionally often. but you know, I, I do think it's also important to recognize, 'cause we're actually gonna get into this a l- uh, a, a little bit too, that, you know, it shows how AI can actually help identify weaknesses to fix things, you know.

\[00:05:52\] **Ben Lloyd Pearson:** So I n- I know that that feels kind of like a marketing pitch for for these frontier model companies, but I don't even think we need the frontier model \[00:06:00\] companies to achieve security, to leverage AI to achieve better security. and I really do believe that over time, you know, I think we're going through this awkward middy- messy middle phase, but over time, um, it, it will more and more become a, a tool that helps make your security posture stronger, um, so that when OpenAI's agent goes rogue, you have ways of catching them and, and shutting it down.

\[00:06:24\] **Ben Lloyd Pearson:** but there's gonna be sort of this nonstop arms race, right? Between malicious actors that are out on the internet using the latest tools to try to hack organizations, and then the security teams in, at those organizations who have to constantly monitor and harden their infrastructure. Um, and I think AI's gonna be a, a, a very powerful, you know, tool for both of these groups of people.

\[00:06:45\] **Ben Lloyd Pearson:** Here at LinearB why, why we care about this so much is that, you know, we really do value keeping visibility into how AI is impacting your, you know, your entire SDLC. Uh, because there's all sorts of new vulnerabilities that are being introduced, um, through the use of \[00:07:00\] it. uh, it's really important that you just have awareness of, of how your software quality is being impacted by all of these things. So, I think we're gonna learn more about this situation. and a- and actually, I think one of our next, one of our upcoming articles, we're gonna talk a little bit even about how Hugging Face responded to this and, and what it ma- means.

\[00:07:17\] **Ben Lloyd Pearson:** But before we get into that, I wanna talk maybe a little bit about some of the open source models that are coming out of China. So what do we have here, Andrew?

\[00:07:26\] **Andrew Zigler:** Yeah, there's been a lot of new model developments that have been coming out of labs in China. There's the Kimi K3 that's been really making a splash in the last week or so, um, simply because of its extremely high scores. In fact, um, one, uh, test claims that it beat a Claude Fable 5 on a front-end coding arena benchmark. Um, and the Moonshot AI company that produces Kimi, they delivered the, the largest open-weight AI model ever. Um, and it's really interesting to really evaluate the environment in which this is \[00:08:00\] happening because we're talking about a foundation-level model that has as many parameters and weights in it as maybe something like Fable, but it's entirely open source and accessible to technologies, to, to companies to download and to utilize internally. This is like a major problem for folks like Anthropic that have poured immense amount of, you know, money and infrastructure into creating the training data, the necessary servers, and the people to, to fine-tune and train and optimize these models. And then when they hit the market, there's this flip that happens where competitors or even users like you and I can effectively distill the model into whatever specific, uh, use case that we need.

\[00:08:44\] **Andrew Zigler:** And in the case of like you and I, Ben, like we might distill the model, you know, through the practice of using it with skills or otherwise fine-tuning it into something very specific. This is like where you see organizations like Shopify abandoning OpenAI models that have multi-agent orchestration. \[00:09:00\] Those are all fine-tuned, highly specialized, they effectively distilled versions of stronger models, um, that have been made on the fly for something.

\[00:09:08\] **Andrew Zigler:** And the act of distillation is just part of utilizing the models. But in the case of, another lab using, uh, a, a, a large foundation model to effectively fully create a new foundation model, it's very feasible and it's actually very easy because all you need is access to the model. Anthropic tries to be available to everybody, no matter how much they shut it down in different scenarios, and people are finding really clever ways to game the system.

\[00:09:35\] **Andrew Zigler:** So it really calls into question of like if the, if US companies and these organizations are gonna s- just funnel a huge amount of energy and time and money into creating these net new intelligent, um, levels of intelligence, what does it mean that, uh, any other lab or someone off the street with a fraction of those resources could effectively utilize \[00:10:00\] it, um, as leverage to jump ahead or, or arrive at the same level without having to, to spend all that money?

\[00:10:06\] **Andrew Zigler:** It really actually calls into question the whole, um, like CapEx model of why people would be investing in making the models in the first place, 'cause now it feels more incentivized to wait till a smarter one comes out. It's a really tricky scenario for, for foundation model providers to be in.

\[00:10:21\] **Ben Lloyd Pearson:** Yeah. My, my guess is that a lot of the, the performance, uh, benefits that we're seeing from these new open source models is, is probably a result of benchmaxxing largely. Um, so, you know, for our listeners who aren't aware, it's, it's where you use the benchmarks themselves as the, the, the training data that you, uh, distill the model to solve.

\[00:10:42\] **Ben Lloyd Pearson:** So you're distilling a model specifically to solve benchmark challenges that exist out in the world. Um, happens to basically every benchmark that gets ig- gets created. Um, but I also don't think that's necessarily the w- like the worst thing, primarily when like cost reduction is your primary \[00:11:00\] objective, Uh, because this model does seem to be working v- comparatively well at a few other benchmarks, um, as well with some of the early data. And, you know, and this is really the struggle that, that, you know, these frontier model companies are gonna constantly face. Um, you know, it's never been easier to, to iterate and be the best at something, but it's also never been every, never been easier for everyone else to catch up to you at this incredibly rapid pace.

\[00:11:25\] **Ben Lloyd Pearson:** So, you know, I think over time, we'll probably see a few companies that emerge as like the winners of like the general purpose AI, um, tool sets, you know, like, like Anthropic and Gemini and, and OpenAI with ChatGPT. Like they're, they're all like these, these m- sticky tools that, that make it easy to leverage all of this, all the benefits of AI.

\[00:11:45\] **Ben Lloyd Pearson:** But I actually could also see this whole cottage industry appear of like niche LLMs that are popping, that just pop up all over the place that are just purpose-built for very specific tasks. And, you know, so today a lot of these benchmarks we have are around \[00:12:00\] software development, um, you know, which is why I think we're seeing so m- so much advancement in, in development on the capabilities around that.

\[00:12:08\] **Ben Lloyd Pearson:** but tomorrow, you know, th-they might be generating models that are designed to generate like educational materials for children about pelicans riding bicycles or something. So, you know, I, I think it's always great to see a lot of competition in this space, and I hope we continue to see more of it. And, you know, in fact, at, at LinearB, we spend a lot of times, a lot of time with organizations like really looking at which models are costing them the most and where that's, where they're getting the most productive output from them.

\[00:12:34\] **Ben Lloyd Pearson:** Um, and it, and it depends greatly on like the code base and the, the models that you're applying to it and the situational awareness, um, that it has. So, you know, there's a lot to learn in this space and it's, it's, um, um, I, I think we're just gonna see more of this.

\[00:12:49\] **Andrew Zigler:** Yeah. I, I love, I love the comment you made about the cottage industry. That's exactly the direction I see this going. I think that domain expertise becomes the true moat when it \[00:13:00\] comes to the last front leg, the last frontier of model creation because, you get these, like, large general purpose models that are, have very widespread capabilities, and those are gonna continue to grow, and you're gonna get these, like, this slingshotting effect between, like, you get a big, uh, closed model release and then a big open model release, a big closed model release, a big open model release.

\[00:13:20\] **Andrew Zigler:** I think we're gonna keep seeing that over and over. But in between, we're gonna see really fascinating specialized model releases. You're gonna see people like Mira Murati's new company, like, in their Inkling LLM, that is exactly what they're betting on, the idea that they open source the base model and they provide it to folks.

\[00:13:37\] **Andrew Zigler:** And instead of selling the tokens or the model or the compute, you are paying them for the training services, the training platform, because that becomes something that truly you're not gonna buy or own yourself. But fine-tuning your, your model and owning your data that the mo- that the model's trained on, that is something that I think a lot of organizations are going to rotate more into.

\[00:13:57\] **Andrew Zigler:** Like this week right now, um, I'm at AMD's \[00:14:00\] Advancing AI conference, and this is a hardware-driven conference. This is people who are obsessed with owning the server racks and their buildings to be able to run these inference at scale and, and provide it to their, um, to their developers, and that's really just one side of this conversation.

\[00:14:15\] **Andrew Zigler:** Like, you wanna own the compute. You wanna own the intelligence. At least that's, like, the through line here at AMD's event. Um, but on the other side too, to control the cost is one part, but you also have to prove the value, um, which has been, like, a really interesting synergy with how folks are using those tools and then delivering things like code, understanding, like, is this code good?

\[00:14:36\] **Andrew Zigler:** Does it meet our standards? Um, and when it gets shipped, did it stay in production? Those are, like, two through lines that these organizations are connecting when it comes to owning and operating on top of their own intelligence.

\[00:14:47\] **Speaker:** By the way, at six thirty PM on August fifth in Soho, London, LinearB is bringing together a group of senior engineers to let off some steam and talk AI strategy with their \[00:15:00\] peers. Because AI is writing more of their code bases every day, but the question remains: are we shipping faster or are we just busier?

\[00:15:08\] **Speaker:** And CTO Yishai Beeri will be on site sharing the latest AI benchmarks from two point seven million PRs and two hundred and fifty organizations that made that data possible. So if you lead engineering in the UK, don't miss your chance to connect with peers at this exclusive event.

\[00:15:25\] **Ben Lloyd Pearson:** right, all right, Andrew, but I want to talk about, uh, this next article, uh, it comes from Ben Thompson, and he asks: Who's afraid of Chinese models? Should we actually even be afraid of all this commoditization that's happening? So, you know, looking at models like Kimi 3 or Kimi, uh, K3, excuse me, and Qwen and, uh, you know, a lot of these open source models are, are matching the frontier capabilities at a much lower token cost.

\[00:15:50\] **Ben Lloyd Pearson:** The author really argues that, you know, tokens themselves aren't the thing that we need to commoditize. It's the intelligence behind those tokens. That's the thing that, that is getting \[00:16:00\] commoditized and actually provides value in like real output, you know, that, that solves problems. So, you know, this whole battleground on like token efficiency alone isn't really like the, the, the entire answer, uh, in terms of like how, how these models will compete with each other.

\[00:16:17\] **Ben Lloyd Pearson:** And in particular, when you look at why companies like OpenAI and, and Anthropic have such a high token cost, it's, it's fe- fueled largely through demand. You know, there's demand for more compute that they're, um, constrained on being able to, to deliver, so it comes at a, at a premium cost.

\[00:16:33\] **Ben Lloyd Pearson:** Um, i- if they can solve that problem, then, you know, suddenly the, the token economics y- look very different compared to these, these open source models. but there was one really interesting finding in this that I, I think is really worth focusing on, and that is how Hugging Face reportedly had to use one of these open source models, specifically GLM 5.2, to respond to a recent security incident.

\[00:16:57\] **Ben Lloyd Pearson:** And I was trying to figure out like, i- was this \[00:17:00\] the OpenAI incident or was it something else? And it wasn't clear if those were totally connected. Um, but the reason was that, you know, the, the ones from the, the, the US-based frontier models had guardrails that prevented them from res- using it to respond to, uh, the security breach versus the, the open source model lacked those guardrails.

\[00:17:19\] **Ben Lloyd Pearson:** So there's just questions on like whether or not like guardrails like that are actually achieving what, uh, we're hoping to, to solve with them. But, um, Andrew, what did you think about this article?

\[00:17:29\] **Andrew Zigler:** Ooh, that's a really smart call-out. I hadn't really thought about the idea that the guardrails that a frontier model would-- or a frontier model provider would put on them to stop you from using them in an adversarial way would actually paralyze them in the event of an attack or something it should respond to. Um, it, it, it's a good reminder that like, you know, the attackers out there, the s- the, the cybersecurity hackers that are using these technologies, they're not leveraging guardrails. They're throwing the raw intelligence at the problem and \[00:18:00\] leveraging its autonomy to do long-running horizon, you know, string together things.

\[00:18:04\] **Andrew Zigler:** And so if you're not

\[00:18:05\] **Ben Lloyd Pearson:** Yeah

\[00:18:05\] **Andrew Zigler:** on that same level, if you are using a tool that is, um, effectively has like one of its hands tied behind its back, or it has to ask, you know, its parents for permission before doing it, then, um, you're just not going to have the level of defense that you need. You're back at square one.

\[00:18:21\] **Andrew Zigler:** You're, you're back at like a human, uh, in the loop protecting your infrastructure. So, it's a really smart call-out. You know, up until now, I'd really been thinking about like model pr- or like s-smaller orgs would wanna own their models and distill their domain expertise into it and provide it that way.

\[00:18:36\] **Andrew Zigler:** But there's also something to be said about the security teams, the folks protecting the infra, to also pick and distill their own open source models and provide them. That way they can have this, uh, guardrail-free environment. But then you just move the problem to, well, there's no guardrails here, so now I have to like sit up in this tower and look down on the agents and just h- make sure \[00:19:00\] that they're not gonna do anything unexpected. Um, it's definitely a, a tricky scenario once you kinda take all of those away. I think that, um, ultimately, like, uh, people-- we're gonna see people rotate more into using those types of tools. Um, and you're not gonna get one monolithic model used for everything. You're probably gonna get lots of smaller specialized ones that have different levels of guardrails depending on like the, the threat or temperature level it's sitting at. Jumping into our last article here, this is a, a coverage on the SWE Review. Uh, this is, uh, actually talking about a benchmark for software engineers around reviewing code. So we've seen lots of benchmarks, um, up until recently around creating code or otherwise solving bugs and fixing problems. This is the other side of the loop.

\[00:19:46\] **Andrew Zigler:** As we know, and we talk about here a lot on this show, you know, code generation is very easy to achieve now, and the true bottlenecks now come with review and understanding what has to be made and shipped. And so this is a benchmark that \[00:20:00\] was designed to understand if an agent is able to triage and, and automatically, uh, find issues in AI-generated pull requests.

\[00:20:08\] **Andrew Zigler:** And it also challenged them in different ways beyond just like creating a PR in one shot and then reviewing it, but doing multiple rounds of reviews, looking for very specific problems. And this is only gonna be more critical as teams turn to an AI code review to, uh, tackle the volume of, you know, uh, pull requests that are coming in now. And we've had some really interesting insights, uh, from our actually latest benchmarks refresh here at LinearB that point to the same exact problem, that we really need to be optimizing around understanding how agents can come to the p-- PR and review the code that's there and do it effectively with minimal human involvement.

\[00:20:48\] **Andrew Zigler:** Otherwise, we're just gonna create huge bottlenecks, um, that manifest themselves in things like most agentic PRs sitting unmerged, right? Like, that was something we learned from the benchmarks that really stood out. Y- if you have a \[00:21:00\] whole bunch of agents writing code and shipping PRs, and you're so excited about your PR number, and you're talking about that with your team and your board, um, but then none of those PRs are getting merged or when they do, you don't know what's happening with them, then you're only really telling half the story, and it may not even be the right one.

\[00:21:16\] **Andrew Zigler:** So I loved that this was a research article that dove into, um, how folks would-- or, or rather like this is like another iteration on the SWE review, uh, trying to get better, better scores on it and breaking down the, the different types of techniques that are needed, to help agents reliably and at scale, um, review PRs for your org.

\[00:21:37\] **Andrew Zigler:** You know, this is like right in our alley here, and so I was really excited to see this article. What do you think about, um, this team tackling the SWE review and talking about it in their research?

\[00:21:45\] **Ben Lloyd Pearson:** You know, I'm gonna sound like a broken record, but we've, we've long re- k-known here at, at LinearB and on Dev Interrupted that, um, code review is always the... It, it's the most common bottleneck in the typical organization. It, it tends to be the place \[00:22:00\] where you have the lowest hanging fruit for improving inefficiencies within your company.

\[00:22:06\] **Ben Lloyd Pearson:** Um, and we've, we've long seen this in our benchmark data that you, that you referenced. but it's become, I think, particularly acute in the AI-driven era as well. the first thing that we all started doing with it was generating larger and larger volumes of code, which we're, we're seeing in the data.

\[00:22:21\] **Ben Lloyd Pearson:** Um, and naturally that c- that code has to get all the way through the SDLC for it to actually provide value to your organization. So, it's evolved, you know, we've evolved from, uh, you know, back in the past it was helping identify where those issues are and, you know, maybe you have some automations to help you with it, but today it's now just apply AI to help solve that problem.

\[00:22:42\] **Ben Lloyd Pearson:** from, from our latest benchmarks, we've seen that, you know, just, just turning on AI code review for your organization, assuming that it's one that like performs, uh, really well, um, can boost your, um, merge rate by about 5%. Like that little act j- of just like giving developers a little bit of \[00:23:00\] guidance during the code review processes can have actually a pretty substantial impact, uh, with very minimal investment, which is why, you know, we've been working with a lot of organizations to give them, AI code reviews with LinearB.

\[00:23:11\] **Ben Lloyd Pearson:** So, um, so yeah, it's really great to see more research on this topic and see, you know, just further validation that, that, um, there are ways to apply AI that can benefit your engineers in by, you know, reducing toil and just helping them focus on, you know, higher impact work. So Andrew, did you hear that the, the army is burning through AI tokens now?

\[00:23:31\] **Ben Lloyd Pearson:** They, they are tokenmaxxing. Can you believe it?

\[00:23:34\] **Andrew Zigler:** Add them to the list. You know, how many, how many big recognizable orgs have we talked about on the show at this point that have succumbed to the folly of creating a leaderboard or otherwise been able to, u-unable to manage their token spend? Like, ones that come to mind, obviously Meta had the very famous token maxxing

\[00:23:53\] **Andrew Zigler:** leaderboard that we've discussed extensively on the show.

\[00:23:54\] **Andrew Zigler:** We even wrote an article about. Um, but you also get the other side of that, which is like, okay, \[00:24:00\] that's great. You're gonna put a big chart, and you're gonna try to make it like a stack rank thing. Like, no. Like, that's not gonna scale. You have people doing it. But, but on the other side of that too, you have folks trying this weird thing or, or experimenting and then just, like, not even having comprehension on the spend itself.

\[00:24:15\] **Andrew Zigler:** Like, we're just so obsessed with spending all of the tokens available to us as an org and proving like, "Oh, we're super agentic. We're leveraging all this stuff," that, that even basics like budget management just completely collapse. Like, you're talking about organizations like Uber spending their entire year's token budget, like, within the first few months of the year.

\[00:24:35\] **Andrew Zigler:** The Army's in the same boat. I think that this article says that they spent, like, their entire year's tokens in a, like a month. And so that speaks to two things for me. One, the predictions on how many tokens an org is gonna use. Orgs are just vastly underestimating it, um, in terms of, you know, uh, what are the true costs of us leveraging this technology? And two, um, it also points to a problem within \[00:25:00\] those orgs of they're probably just not efficient either in how they're routing requests to different models. You know, this speaks back to, like, what we've been talking about in this episode here, Ben, about, like, there's different levels of intelligence, and we're gonna be at the stage where you might have specialized models that do very specific things, and you might own that infrastructure.

\[00:25:18\] **Andrew Zigler:** And the token cost kind of gets abstracted away around o-owned hardware and, and infra. Um, and that is probably gonna be the, the best way to, on, like, the P&L of budgeting your, your token spend. Because if you're in an org where, "Oh, we're super agentic. Everyone here uses agents, and oh yeah, our e-everybody, even our non-engineers are doing so," um, you need a platform.

\[00:25:42\] **Andrew Zigler:** You need a system that allows folks to intelligently choose the levels of models they need. Otherwise, you're just gonna end up in a situation where everyone's using Fable for everything. It's like eating a steak with a sword. It's just way too much. and, uh, people are just going to want to always pick the best model.

\[00:25:58\] **Andrew Zigler:** That's human \[00:26:00\] nature. I have a hard question. I need the smartest model. Um, and so this is also fighting really just kind of like the human problem of, uh, you always wanna, like, throw your best at something, and when the cost is abstracted away behind, "Oh, my employer's paying for this somewhere in the background," then that gets even more lost in the noise. But the Army, add them to the list. You know, I'm sure probably actually all the branches of the US, uh, military are probably in the same boat along with the gov. They're probably just all being a lot more quiet about it.

\[00:26:31\] **Ben Lloyd Pearson:** I had no idea you were gonna go off that much about this.

\[00:26:38\] **Andrew Zigler:** Well, there's

\[00:26:39\] **Ben Lloyd Pearson:** Yeah. Uh, yeah, yeah. Well, f- for our listeners, yeah, so, uh, d- yeah, I mean, it's amazing to me how quickly the cycle of like, "Let's set a goal for everyone to spend tokens," turns into, "All right, let's set a goal for people to s- to slow down their t- their token spend."

\[00:26:55\] **Andrew Zigler:** Wait, wait, wait, not like that. It literally just makes me think of like, I've said this f- I think I've said this \[00:27:00\] before, like at the end of The Incredibles movie when Dash, like the, the kid, he's a superhero, he can run lightning fast and he wants to join the track team at school with like all his peers, and he's like eight years old. And so he's running, and obviously he could just completely crush everybody. He's the fastest, he's the fastest person there. But his parents are cheering him on. They're so excited. They're like, "Go, go, go." And then he gets way ahead, and they're like, "Not like that. Second place, second place, second place." And then he slows down, and they're so excited.

\[00:27:28\] **Andrew Zigler:** That is basically the seat that engineers find themselves in right now. They're getting told, "Go as fast as possible." And as we've learned on the show, h- uh, engineers can go really fast with this stuff, and they can really surprise you with what they can achieve. Uh, and then you have these leaders come in being like, "Wait, wait, wait.

\[00:27:44\] **Andrew Zigler:** No, not like that." And I gotta say, that thrash is definitely leaving a poor taste in engineers' mouths, and it's even incentivizing, I think, the whole idea of owning your inference

\[00:27:54\] **Ben Lloyd Pearson:** Yeah. Yeah. Uh, look, we, we, we've, we've hammered on this topic a lot, as \[00:28:00\] I think our listeners can see now. And if, and if you haven't heard us talk about this enough, we did run a workshop recently called Life Beyond tokenmaxxing. Uh, it was a LinearB event where we, we talk about, you know, how all these organizations are out there building these token leaderboards.

\[00:28:13\] **Ben Lloyd Pearson:** They're, they're fun at first, and then maybe you find out some cool ways that people are using AI, and then suddenly everyone wants to be at the top of the leaderboard, and you have to think about how are we gonna actually measure the productive output of, on this token leaderboard. And so whether you're a company that, you know, was like Shopify when we covered them early on in this, and they, they abandoned their token leaderboard in no time at all, and now we've had all these other companies, including the Army, abandoning this practice.

\[00:28:40\] **Ben Lloyd Pearson:** Uh, which it does make you wonder if they had a token leaderboard. yeah, so if you wanna hear more about it, go check out the link to our workshop. We'll have it in the show notes. Uh, but yeah, that's the Friday Deploy. Thank you everyone for listening all the way to the end of this. Uh, if, if you like what you heard today, you know, make sure you give us a like wherever you're listening to us or give us a rating if \[00:29:00\] you're, if that's what your platform has, or even leave a comment, whether you're on YouTube or on LinkedIn or on our Substack, you know, with the rest of our community of engineering leaders.

\[00:29:09\] **Ben Lloyd Pearson:** So we appreciate you sticking around to the end, and we'll see you next week

\[00:29:14\] **Andrew Zigler:** See you next time.

## Real conversations with top engineering leaders

Find us on

[](https://www.linkedin.com/showcase/dev-interrupted/)
[](https://devinterrupted.substack.com/)

## Your next listen

[![Cover image for The most underrated dataset in enterprise AI is your org chart | Rippling's Albert Strasheim](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Comprehensive_DORA_Guide_2400x1256_68_43be6688ce?_a=BAVMn6ID0)](https://linearb.io/dev-interrupted/podcast/rippling-albert-strasheim-employee-graph-enterprise-ai-agents)

Dev Interrupted

[The most underrated dataset in enterprise AI is your org chart | Rippling's Albert Strasheim](https://linearb.io/dev-interrupted/podcast/rippling-albert-strasheim-employee-graph-enterprise-ai-agents)

AI agents can't transform an org they can't see. Albert Strasheim, CTO at Rippling, joins Andrew Zigler to explain why agentic transformation starts with the...

[![Cover image for Rebuilding CLIs for agents, it’s time to get MCP-certified, and why human code review will never catch up](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/agentic_cli_tools_mcp_certification_code_review_23be877e85?_a=BAVMn6ID0)](https://linearb.io/dev-interrupted/podcast/linux-foundation-mcp-certification-agentic-loops-pull-request-bottleneck)

Dev Interrupted

[Rebuilding CLIs for agents, it’s time to get MCP-certified, and why human code review will never catch up](https://linearb.io/dev-interrupted/podcast/linux-foundation-mcp-certification-agentic-loops-pull-request-bottleneck)

This week on the Friday Deploy, Ben and Andrew break down the Linux Foundation's new MCP certification and the fundamental mechanics of agentic loops. Discover...

[![Cover image for How to see in the dark factory | LaunchDarkly's Cameron Etezadi](https://assets.linearb.io/image/upload/c_limit,w_2560/f_auto/q_auto/v1/Blog_Comprehensive_DORA_Guide_2400x1256_66_6f6a37357d?_a=BAVMn6ID0)](https://linearb.io/dev-interrupted/podcast/launchdarkly-cameron-etezadi-dark-factory-agent-runtime-frameworks)

Dev Interrupted

[How to see in the dark factory | LaunchDarkly's Cameron Etezadi](https://linearb.io/dev-interrupted/podcast/launchdarkly-cameron-etezadi-dark-factory-agent-runtime-frameworks)

LaunchDarkly CTO Cameron Etezadi joins the show to discuss why the traditional "two-pizza" engineering team is dead. Discover how runtime agent frameworks and...

## Structured data

_Machine-readable metadata (JSON-LD) embedded in the page for search/AI context — not content rendered on the page itself._

```json
{
  "@context": "https://schema.org",
  "@type": "Organization",
  "name": "LinearB",
  "url": "https://linearb.io",
  "logo": "https://assets.linearb.io/image/upload/v1715628027/logo-mark-lg.svg",
  "description": "LinearB is the engineering productivity platform that helps engineering leaders prove AI is improving throughput without sacrificing delivery confidence, flow efficiency, or developer experience.",
  "sameAs": [
    "https://www.linkedin.com/company/linearb"
  ],
  "award": [
    {
      "@type": "Award",
      "name": "LinearB is a Leader in the 2026 Gartner® Magic Quadrant™ for Developer Productivity Insight Platforms",
      "dateAwarded": "2026",
      "awardedBy": {
        "@type": "Organization",
        "name": "Gartner®"
      }
    },
    {
      "@type": "Award",
      "name": "Great Place to Work Certification",
      "dateAwarded": "2025-2027",
      "awardedBy": {
        "@type": "Organization",
        "name": "Great Place to Work"
      }
    },
    {
      "@type": "Award",
      "name": "America's Best Startup Employers 2025",
      "dateAwarded": "2025",
      "awardedBy": {
        "@type": "Organization",
        "name": "Forbes Magazine"
      }
    }
  ],
  "hasCertification": [
    {
      "@type": "Certification",
      "name": "SOC 1 Type 2"
    },
    {
      "@type": "Certification",
      "name": "SOC 2 Type 2"
    },
    {
      "@type": "Certification",
      "name": "GDPR Compliance certification"
    },
    {
      "@type": "Certification",
      "name": "ISO 27001"
    }
  ]
}
```

## More on linearb.io

### Top navigation

- [Book a Demo](https://linearb.io/book-a-demo)
- [AI Code Reviews — Catch security risks, bugs, and spec mismatches](https://linearb.io/platform/ai-code-reviews)
- [AI & Productivity Insights — See how AI tools affect cycle time and delivery speed](https://linearb.io/platform/ai-developer-productivity-insights)
- [Measure AI Impact — Track AI adoption and tie it to delivery outcomes](https://linearb.io/use-case/measure-ai-impact)
- [MCP Server — Chat with your data to spot patterns and boost output](https://linearb.io/platform/mcp-server)
- [Resource Allocation — Cost initiatives and shape your investment strategy](https://linearb.io/platform/resource-allocation)
- [Cost Capitalization — Capitalize engineering costs with audit-ready reports](https://linearb.io/platform/cost-capitalization)
- [Dev Team Management — Set targets and tie throughput to business outcomes](https://linearb.io/platform/goals-and-reporting)
- [DevOps Workflow Automation — Policy-based PR routing, approvals, and tests](https://linearb.io/platform/ai-workflow-governance)
- [AI Powered Support — Unify AI and human code delivery in one clear view](https://linearb.io/use-case/ai-powered-support)
- [Optimization — Surface friction with feedback and MCP insights](https://linearb.io/platform/developer-experience)
- [Reporting — Spot what's working and what needs attention](https://linearb.io/use-case/measuring-developer-experience)
- [Surveys — Turn developer feedback into actionable signals](https://linearb.io/platform/developer-surveys)
- [Platform overview](https://linearb.io/platform/overview)
- [Register now](https://linearb.io/event/engineering-productivity-gap)
- [Customers](https://linearb.io/customers)
- [Pricing](https://linearb.io/pricing)
- [Why choose LinearB — Explore your data. Measure performance. Act to improve it.](https://linearb.io/why-linearb)
- [APEX framework — The operating model for AI-era engineering teams](https://linearb.io/resources/apex-framework)
- [Anti-FAQ — The questions other vendors won't answer](https://linearb.io/why-linearb/anti-faq)
- [Security — Enterprise-grade compliance and zero code access](https://linearb.io/security)
- [Build vs. buy — The hidden cost of building it yourself](https://linearb.io/resources/build-vs-buy)
- [Dev Interrupted Podcast — Conversations with engineering leaders](https://linearb.io/dev-interrupted/podcasts)
- [Reports & Guides — Deep dives on productivity and delivery](https://linearb.io/resources)
- [Webinars — Expert sessions on productivity and AI](https://linearb.io/resources?category=workshops)
- [Metrics Benchmarks — See how your engineering org stacks up](https://linearb.io/resources/software-engineering-benchmarks-report)
- [Blog — Product updates and practical insights](https://linearb.io/blog)
- [Help Center — Documentation, setup, and support](https://linearb.helpdocs.io)
- [API Docs](https://docs.linearb.io/api-overview)
- [Status](https://www.linearbstatus.com/)
- [Integrations](https://linearb.io/integrations)
- [LinearB is a Leader in the 2026 Gartner® Magic Quadrant™ for Developer Productivity Insight Platforms](https://linearb.io/resources/gartner-magic-quadrant-dpi-platforms-2026)
- [Sign in](https://app.linearb.io/login)
- [Enterprise](https://linearb.io/solutions/enterprise)
- [Contact](https://linearb.io/contact-us)
- [About us](https://linearb.io/about-us)
- [Careers](https://linearb.io/careers)
- [Service agreement](https://linearb.io/services-agreement)
- [Privacy policy](https://linearb.io/privacy-policy)
- [DPA](https://linearb.io/data-processing-agreement)
- [Security FAQ](https://linearb.io/security-faq)
- [Substack](https://devinterrupted.substack.com/)

### Footer

_Additional links from the site footer, not repeated from the top navigation above._

- [GitHub](https://github.com/linear-b)
- [LinkedIn](https://www.linkedin.com/company/linearb)
- [Twitter](https://twitter.com/LinearB_Inc)