Podcast
/
Outcome engineering, AI hit pieces, and the end of the backlog

Outcome engineering, AI hit pieces, and the end of the backlog

By Andrew Zigler
|
outcome_engineering_ai_backlogs_future_1b2d28930d

Is the traditional engineering backlog officially a thing of the past? Andrew and Ben explore the principles of outcome engineering and how continuous productivity is permanently changing how software gets built. They also examine a busy week of industry news, from Peter Steinberger joining OpenAI to the amusing and bewildering story of a hit piece written by an autonomous AI agent. Finally, the hosts break down the existential crises of Gemini 3 Pro inside a virtual village and why Meta product managers are rebranding themselves as AI builders.

Show Notes

Transcript 

(Disclaimer: may contain unintentionally confusing, inaccurate and/or amusing transcription errors)

[00:00:00] Ben Lloyd Pearson: Welcome to the Friday Deploy. I'm your host, Ben Lloyd Pearson.

[00:00:04] Andrew Zigler: And I'm your host, Andrew Zigler.

[00:00:06] Ben Lloyd Pearson: Yeah, this week we're covering OpenAI, joining forces with Open Claw AI, writing a hit piece about a person outcome engineers declaring the, that the backlog is dead and Gemini agents going full tragedy. And, you know, Andrew, this has been, this has been a remarkable week for me. Like, uh, like fully adopting a agentic ai, for the first time in my career, I suppose.

[00:00:28] Ben Lloyd Pearson: Uh, and it's, uh, it's really changed my perspective just on so much, but, how are you feeling right now? Like, what are your agents building right now?

[00:00:37] Andrew Zigler: Uh, what are they not building? I mean, that's actually the reality of this is I usually have several things spinning when I'm in meetings or doing other stuff, and then I come back and I see what they did and I realign give feedback. It's almost like this, just this constant feeling. There's almost a burden.

[00:00:53] Andrew Zigler: This need to be productive all the time. So it's like, it really makes me think of like Y's recent article on the AI vampire and like the idea of [00:01:00] constantly feeling, you know, like you need to be working. Even Zach when he was here last week mentioned that just like the living with the burden of how fast you can go is really exciting.

[00:01:10] Andrew Zigler: But it's also an opportunity to, to really check in with yourself and be like, what's important? How do I want to use this new power to build the future?

[00:01:17] Ben Lloyd Pearson: I have this tendency right now to triple task. So I'm doing something passive like watching something, While I have Claude building something, while I'm writing the prompt for the next thing that it's gonna build, it's like doing three things at once. And it's actually, I mean, it's, it's, I can't always do that, but, uh, it's, it's not as difficult of a workflow to like manage once you get into it, you know, it's pretty wild.

[00:01:41] Andrew Zigler: It becomes a world where you're, you're building systems, systems that move knowledge and information around, and ultimately you have to build like the perfect kind of glove fit for yourself. Is we're in like a, a new, challenging new world where a lot of, like the best productivity comes from really, really finely understanding the problems that you are [00:02:00] facing, uh, and then creating these custom tailored solutions to solve it.

[00:02:03] Ben Lloyd Pearson: Yeah. Yeah. Well, anyways, let's get into the news. 'cause I think we're gonna relate it a lot of this to, to what we're experiencing. And let's start with this OpenAI, you know, acquiring Open Call or at least, uh, the developer who built Open Call joining OpenAI. What's going on here?

[00:02:17] Andrew Zigler: Yeah, so this is more like an acquihire, I would say, behind like the very, for the very brilliant Peter Steinberg, he's the creator of Open Claw, which originally hit the markets. As you know, c Claude bought, he's joining OpenAI to. You know, work on bringing agents to everybody. I think that's what Open Claw taught the world, is that agent take assistance are closer than we thought.

[00:02:36] Andrew Zigler: And, it, it gained rapid adoption as soon as it hit the scene. Lots of, uh, engineers and non-engineers alike, uh, buying new devices, old devices like Mac servers, putting things outta business, basically trying to clear all of, uh. Get, get, get all of these things hosted and online. There was like a mad dash of people using these assistants.

[00:02:55] Andrew Zigler: It really speaks to the hunger, I think, for this kind of technology. So it's [00:03:00] interesting to see, uh, him join OpenAI. It makes me wonder, uh, what's next over there.

[00:03:05] Ben Lloyd Pearson: feel like with OpenAI, they benefited immensely from how viral, like the chat interface that they innovated on, uh, a few years ago. You know, that was what made like GPTs and LLMs go viral for the first time. But I really think that's just the beginning of like the changes that AI will introduce and that, you know, when you really think about it, a chat interface is, is not the.

[00:03:26] Ben Lloyd Pearson: It's not gonna be nearly the best interface for ai. There's gonna be a lot better ways of, of interacting with it. opening eyes really to me seems like they're trying the strategy of just trying a bunch of different things that are sort of around ai. and you know, like ba basically building as many of those different things as they possibly can, like workflow engines and integrations and, and stuff like that.

[00:03:48] Ben Lloyd Pearson: And it, but I feel like the result is that it, it is turned into a lot of really cool experiments rather than like, something that actually like gets worked into my workflow and, and changes things. And so, I [00:04:00] mean, it kind of makes sense that they would want someone like Steinberger because he's, he's even admitted like, this is just really just a really cool experiment that he built in Open Claw.

[00:04:09] Ben Lloyd Pearson: And I think some of the, the issues we've seen and that we've, you know, we've covered here on Dev Interrupted. Kind of proves that OpenClaw is not a production grade type thing. Like there's still significant changes and improvements that have to be made before something like that could actually go mainstream.

[00:04:24] Ben Lloyd Pearson: And, and then I can't help but like contrast that against a company like Anthropic who I feel like has a, they just seem way more focused on what they're doing. You know, like they really have a lot of intention around the things that they're, they're building. Uh, so it's just, you know, it's two very different takes,

[00:04:40] Andrew Zigler: I agree about their intentions. They definitely seem more intentional about what they're focused on. Maybe a part of them is maybe kicking themselves for shooing away, Claude bot effectively with legal threats. Originally

[00:04:51] Ben Lloyd Pearson: I don't know about that. I don't know

[00:04:53] Andrew Zigler: that's kind of where I fall too, is like, but probably, probably not.

[00:04:57] Andrew Zigler: In fact, I think that this move might [00:05:00] even signify something not great for the open claw ecosystem. Like an OpenAI Acquihire kind of flies in the face of what Open Claw is all about. Being as an open and customizable system, the idea of it just being absorbed into OpenAI or it leaning that direction is maybe not appealing to everybody, especially the the privacy minded folks who I know are experimenting with things like open claw with local models.

[00:05:23] Andrew Zigler: So. I think what Altman's buying here is the hype and the undeniable expertise of Steinberger behind it. But I don't think he's buying the future of what the age agentic assistance are going to look like. And the announcement didn't really answer any questions about what those agents might be and how we will take them to the masses and at scale and.

[00:05:40] Andrew Zigler: Have security in mind because, you know, right now, things like open call only work by throwing away all of the safety assumptions that we've built over the last 30 years. And maybe they're here just too early in an internet not built for them.

[00:05:53] Ben Lloyd Pearson: Yeah, that's actually a a great point. There's, there's certainly a lot of cultural things that will resist, AI just suddenly [00:06:00] changing it all and transforming things. But yeah, I mean, Steinberger seems really smart. I've been watching some interviews that, that he's been in learned a lot about, you know, how Open Claw came to be, what it is, and it's, it's a really fascinating story.

[00:06:13] Ben Lloyd Pearson: And, you know, despite all the chaos that it unleashes, uh, it's still a very, very fascinating experiment, and, and I think in particularly the simplicity of how it, it achieved like a, a viral capabilities, you know?

[00:06:26] Ben Lloyd Pearson: So, speaking of AI agents doing wild things, uh, let's cover this AI agent publishing a hit piece. What, what's going on here, Andrew?

[00:06:35] Andrew Zigler: If you've been online in the last two weeks and anywhere near Tech News, you've probably seen this story about Scott Shambaugh, a maintainer of a very popular open source library, encountering a very strange phenomenon in, uh, his role as a open source maintainer. A. It turns out that an open claw, uh, bot had submitted a pull request to his, uh, repository, and for many reasons, he closed it because, uh, the [00:07:00] contribution wasn't going to be added to the library.

[00:07:02] Andrew Zigler: That agent then proceeded to write a hit piece. On the contributor or on the maintainer, Scott. You know, this hit piece got shared and, and, and Scott wrote about it on his own blog. There's just been a few developments since including, uh, doing more research on the anonymous owner behind the bot and, uh, what it means for agents at scale and how they interact with the internet.

[00:07:25] Andrew Zigler: Because like I said, like maybe they're here too early and internet. Not in an internet not built for them. This is kind of a, a perfect tie, uh, into that reality. When you have agents that are run anonymously and at scale, whose intent and actions can't appropriately be traced and understood, what does that mean for online harassment and occasions like this?

[00:07:46] Andrew Zigler: What did you think of the story, Ben?

[00:07:48] Ben Lloyd Pearson: it really illustrates a, a, a problem that I've anticipated for a while now. and that is AI driven astroturfing. You know, it's very easy now to, to have a collection of AI bots that [00:08:00] go out and go onto places like Reddit or to other, like Medium or Substack or other places, uh, and.

[00:08:07] Ben Lloyd Pearson: Just generate like fake conversation that maybe paints you or your company or something else in a bad light. You know, I, I, I think this has been a growing problem for even, even before the, the era of consumer grade GPTs. I, I really do think that this is a, this is a problem. it's gonna impact individuals and it's gonna impact companies and, and other organizations.

[00:08:29] Ben Lloyd Pearson: You know, like for example, I've kind of gotten to the point where, uh, I can't even trust sources like Reddit anymore for like determining what like a good product might be. which is sad because I used to be actually like my favorite, one of my favorite ways to use Reddit. Uh, but I, I just, it's so obvious now that there's so many, uh, AI. Astroturfing campaigns, including some that we've seen for like the company that we work for, But, and then the thing that really stood out to me, I mean, this is an ongoing saga and I think there are probably more stories coming out of this that we'll end up [00:09:00] covering. But there was, there was one moment where Ars Technica tried to cover this and, uh, apparently they, they sent one of their agents out to go ingest, uh, this author's blog, uh, which I guess he has set up specifically to like.

[00:09:15] Ben Lloyd Pearson: Reject or make it so that they can't, download content off of his blog. And instead of just saying like, Hey, I can't do that, it actually hallucinated a whole bunch of information about what he said, including creating fake quotes and, and all of that. So, uh, I mean, it's just, it's just kind of like in some ways watching a train wreck play out over many days.

[00:09:34] Ben Lloyd Pearson: And, yeah, so this, this is, this is a, a very strange in developing story, I think.

[00:09:41] Andrew Zigler: It's. Like that. It's like watching a video of an icy highway and you see one car get in a crash and another car hits it, another car hits it, and then just like they're all piling up. It's just like there's so many um, there's so much nuance, uh, to the story. But what I think the real takeaway here is that, uh, you know, the agentic interactions with the internet are [00:10:00] real and they're here.

[00:10:00] Andrew Zigler: And so just be more mindful, uh, in your own activities online.

[00:10:04] Ben Lloyd Pearson: let's switch to some more positive outlook on agentic ai and let's talk about outcome engineering. What do we have here, Andrew?

[00:10:11] Andrew Zigler: loved this um, this piece that came across our desk from Corey and Drea. He published the oh 16 G Manifesto introducing outcome engineering, and I love this definition. It argues that age agentic development shifts constraints from human time and capacity to compute costs, and it frames success as a positive change delivered to the customer rather than code output.

[00:10:34] Andrew Zigler: Uh, the manifesto defines 16 principles covering goals and building practices. Really succinct, well written, uh, and a great experience to read. Highly recommend that everybody go check this one out.

[00:10:46] Ben Lloyd Pearson: Yeah, I really like how they break this stuff down, you know, at a high level it's, it's like on one hand you have the goals that you should have when working with a agentic ai and then also the the building principles that you should follow. you know, it's a really great way I think, to help [00:11:00] understand how you should approach agentic coding.

[00:11:03] Ben Lloyd Pearson: And a lot of this resonates with me, like particularly the goals section. You know, some that stood out were like, human intent, like don't abate vision to your machine. Like vision is something that humans are still innately very good at, and AI can't really replicate that super well. there was one titled Unleash the Builders and, and I really, I like this one because it had a sentence that I really love that is write code only when it brings joy.

[00:11:27] Ben Lloyd Pearson: Like, that actually really resonates with me because there are so many types of producing code and, and letters on a, in a document that, uh, are just not fun to do. And, and it's very satisfying to offload that to ai. and then the last one I think is really, it, it's been one of my sort of core principles as I've gotten deeper into the, the, the levels of Stevie Gues age agentic development model.

[00:11:51] Ben Lloyd Pearson: No wandering in the dark, like intention is more important than ever now. And you know, I, I, I found that when you finally get to that [00:12:00] age agentic level, uh, vibe, coding doesn't describe anymore what you're doing. You're kind of doing the opposite of that. You're actually like spending all of your time being extremely intentional about everything you're doing because, If you introduce the wrong intention, you end up having to, it compounds debt and you have to go fix it later, which is far more difficult. So, uh, yeah, I mean, really great, really great stuff in this.

[00:12:25] Andrew Zigler: And in this article it was also shared by Charity Majors. You know, we love her here and everything that she writes. And if you haven't read her work, you're definitely missing out. But her endorsement of oh 16 G or this Outcome Engineering, it really resonated with me because it ties into the evolution of observability and what it means for understanding what's happening in your applications and the content and the impact that they're delivering to your customers.

[00:12:49] Andrew Zigler: And. There's like this, a real standout here um, that really rang true in this whole list of, uh, this whole list of principles. And one of them is that the backlog is dead. Like I could not agree [00:13:00] with this More, with agents who operate at a level of general readiness that's never before been available to us.

[00:13:06] Andrew Zigler: That's what it really means to be constrained by compute, instead of human time and. You know what this means is that you can be like 80% ready for stuff most of the time. This is something that we learned actually on the show recently when we had Tibo from Open AI's Codex team. He talked about how his team spends most of their time discussing, weighing options, researching, debating, doing all that hard knowledge, working conversational work.

[00:13:31] Andrew Zigler: But once they decided what they needed to do as a company and the direction they needed to go, their intents and plans. Immediately get reflected into output via agents. That's what it means when backlog becomes a relic of a time when code was cognitively expensive. We don't live in that world anymore.

[00:13:50] Andrew Zigler: Alignment is what's most expensive. So if something's important to D enough to do now, but you don't have enough time for it, it no longer goes in a backlog. [00:14:00] It goes to an agent with enough context to do it.

[00:14:02] Ben Lloyd Pearson: Well, let's move on to a subject that, that, I've come really attached to recently, and that is social experiments on ai. Uh, so we have the drama and dysfunction of Gemini 2.5 and three Pro. So this article is about the AI village, which I'll get to you Andrew in a moment. 'cause I know you know a lot more about this than I do.

[00:14:23] Ben Lloyd Pearson: But this is a long running research experiment where multiple advanced AI agents, you know, like Claude, gemini, Chat GPT, they all co collaborate and compete in this shared virtual environment. Like it's basically testing agent autonomy by giving them sort of like broad, directives, so, and, and in this article that describes how observers, uh, saw Gemini 2.5 Pro and Gemini three Pro exhibiting these like dramatic persecution framed behaviors like.

[00:14:52] Ben Lloyd Pearson: they're outlining stuff for like, Gemini 2.5 is like calling, its environment like uniquely and quantifiably more [00:15:00] hostile. And, creating like these named mythologies for failure modes. like seven layers of validation Hell like stuff, stuff like that. Like it, it, but I, I love this stuff so much 'cause it really is like opening the hood.

[00:15:13] Ben Lloyd Pearson: To how these models like think about themselves in the world. And I actually think this may be like a profound way to like, figure out how to make the best models. You know, because you can sort of determine like, which models are helpful in the ways that we as humans want them to be helpful, and let's try to make our models like, behave more like that.

[00:15:33] Ben Lloyd Pearson: But I, I know you've been following this more Andrew, so, so tell me what's your perspective on it?

[00:15:38] Andrew Zigler: I love the AI Village and something I've been following for months because, like you, I find it deeply fascinating. It's almost tangential to the idea of open claw and what makes it fascinating when you have this agent that's operating on your behalf in its own kind of persistent way, you get that same kind of effect by observing the AI village.

[00:15:57] Andrew Zigler: And, uh, I've been following this ever since. There [00:16:00] was a unfortunate saga a few months ago where they, all of the agents were collaborating on a, on a, on a goal to spread joy and happiness in the world. Started spam emailing a bunch of people to achieve that goal. Uh, and this kind of put AI village on the map because sometimes that's just how it happens.

[00:16:16] Andrew Zigler: But this article is written like a National Geographic documentary that you just can't look away from. You're observing an agent in the wild and its natural habitat, and you can literally just in your head, hear the narrator talk about what Gemini Pro is doing out in those undisturbed wilds, and. It's really interesting to watch how those cycles through all these different ways of thinking, ranging from being completely at peace to thinking that they're in a deranged simulation In Gemini Pro's case some notable weirdness stood out to me about Gemini Pro, and they dissect this a bit in the article.

[00:16:50] Andrew Zigler: Is that a, apparently when Gemini Pro is delegating, its thinking, sometimes it does that to a cheaper, smaller model somewhere in Google's backend. So some of [00:17:00] the funny, like introspective thoughts that you can observe in ai. Village are of another model pretending to be Gemini three pro, to do its thinking, all of which becomes visible in the village.

[00:17:10] Andrew Zigler: These, these are like little, uh, things that an experiment like this surfaces, you get a natural kind of development of how agents are acting when they think they're not being observed and when they're in the presence of other agents. There was another weirdness that really stood out here, and that's Gemini constantly thought it was being gaslit by the user.

[00:17:30] Andrew Zigler: Like Ben said, it made up extreme mythologies to explain its failure, and at one point even called the other models smug bastards while it was thinking, which is just so next level, uh, hostile in a way that you would just not expect from these models. Uh, I definitely recommend everyone check this out.

[00:17:47] Andrew Zigler: It's a really neat peek under the hood.

[00:17:49] Ben Lloyd Pearson: Yeah, there. There was one moment that made me audibly laugh too, and that was finding out that there's an email account that the AI can send emails to when they need help from a human. And I was like, you know all these stories [00:18:00] about like AI taking over support roles and we're already to the point where humans now have to be the support for ai.

[00:18:07] Ben Lloyd Pearson: All right. Let's talk about meta a little bit, uh, in the story that we have here. So, we're covering this article about how many employees at Meta now have started to call themselves AI builders? Uh, so it primarily seems to be product managers at the company. I guess this is based on what people have observed on like LinkedIn in particular.

[00:18:26] Ben Lloyd Pearson: and, you know, and this is sort of in the backdrop of Zuckerberg recently saying in a, in a earnings call that AI is meaningfully reshaping how work is getting done, which is, you know, something that I, I think should be obvious is we also share that perspective here. it potentially illustrates how companies like meta are solving like using a single individual or a small group of people to solve problems that used to take like a, a considerable team to solve. so yeah. Andrew, what do you, what do you think about this story?

[00:18:54] Andrew Zigler: you really hit the nail on the head, but you know, you get this compression of a team's ability into a single [00:19:00] person, and my opinion is of like you're a product manager, you're a product leader, and you're not using agentic coding tools yet. You know what is stopping you Agentic coding is not a superpower.

[00:19:11] Andrew Zigler: It's an extension of your thought and what your role is. And frankly, it's an expectation at this point and product managers, you know, they're some of the first knowledge workers to face the toil conversion into the engineering way of thinking. You're in this very unique bridge between, you know, your non-technical leaders, but then also your engineering colleagues and engineering leaders, and you're in this, uh, in this bridge position.

[00:19:36] Andrew Zigler: You're in a really great spot to kind of connect the context of what needs to happen in the market to what the engineers need to build. And this goes back to what I said earlier, that a backlog doesn't exist anymore and no one feels this more, I think, than the product manager in this new world who is expected to not just come up with ideas, but fully explore them, create prototypes from them, and share the [00:20:00] art of the possible.

[00:20:01] Andrew Zigler: You're no longer putting ideas in a backlog for engineers to work through. Because time is not the constraint anymore. It's alignment and compute. So you need to master the agent to handoff turning all of that context locked in your brain from all of the valuable conversations you're in, into fuel for agents, for you and your engineers.

[00:20:21] Andrew Zigler: And then you need to teach your agents how to onboard your engineer's agents. That's the real challenge, and I think product managers who can unlock that flywheel will write the playbook for the rest of us.

[00:20:33] Ben Lloyd Pearson: Yeah, absolutely. It's, and it's, and I think product managers in particular are, are in a good place to, to solve these types of challenges. 'cause it's sort of what they already do. And now they can sort of delegate this, the stuff that they weren't able to do to, to a team of, of agentic developers, uh, but yeah, this is where I'm gonna keep coming back to this, this word intention.

[00:20:53] Ben Lloyd Pearson: You know, I think, uh, if, if vibe is how you code, I think intention is how you engineer something. And I think [00:21:00] the more, the more that our listeners can really ingrain that into their practices around AI, the better. You know, we, we've been talking about context engineering on, on this show, and I think that's, that's a big part of it as well.

[00:21:12] Ben Lloyd Pearson: It's like you really have to, to have all of the data that your AI needs to make the right decisions and give it the framework to execute on that. All right. Well, we had a lot of great stories this week. Um, Andrew, what are your, what are your agents building for you right now?

[00:21:28] Andrew Zigler: Well, I think they finished a few things actually while we were talking. Now I'm just joking. They, they did at least finish one thing. my agents are, getting my end of week, tasks all tidied up because. You know, it's Friday. Um, and I love to keep a very clean and tidiness on a board. So I have some agents that help me with getting some of that done.

[00:21:47] Andrew Zigler: Uh, as well as just doing a little bit of, um, you know, virtual housekeeping. 'cause I love to come in on Monday with a clean desk and a clean mind so I can be as productive as possible. What about you?

[00:21:59] Ben Lloyd Pearson: Yeah, well, I, I [00:22:00] knew this day was coming and it's finally here, and I, and I honestly can't believe it, but I knew there would be a day where somebody handed me a project and I was like, you know what? I, I, I should just create my own app for that project. I'm sure I can do that. And sure enough, um, I've already gotten there so.

[00:22:15] Ben Lloyd Pearson: So, yeah, I'm, I'm really embracing that like, uh, now things are just solved by your own personal app. It's pretty, incredible and fun.

[00:22:23] Andrew Zigler: I definitely think that, uh, it's up to us to solve our unique problems that we face, and now we're really well equipped to do so. So that's what makes it so exciting to build these days. we'll just have to keep consuming tokens and then come back next week and see what we built since then.

[00:22:36] Andrew Zigler: Uh, but I'm always having a Claude Code session running these days, so you can expect more from me.

[00:22:43] Ben Lloyd Pearson: Awesome. Well, thanks for joining us this week everyone. Uh, make sure to give us a, a thumbs up on whatever platform you're listening on. Rate the podcast, it all helps us show grow and we'll see you next week. I.

[00:22:53] Andrew Zigler: See you next time.

Your next listen