“… even before we get into any of the details of specific things that might've failed with DevSecOps, the incentives just don't align between the security team and the  development team.”

If you're tired of hearing "shift left" in DevSecOps and seeing little real change, you're not alone.

In this episode, David Mytton (CEO of ArcJet, founder of Console.dev) breaks down why traditional approaches to developer security often fail. He reveals the core conflict between developers (who want to build fast) and security teams (who want to mitigate risk), and explains why this misalignment of incentives can be detrimental for your software. Learn why simply handing devs more security tools isn't enough.

David shares his insights from years of experience reviewing developer tools and building security products. He discusses the importance of developer-centric design, the power of the right incentives, and the need for security solutions that seamlessly integrate into the developer workflow. Plus, he reveals the secrets to successful developer marketing and why traditional approaches often backfire.

Tune in to discover how to foster a security-conscious culture within your engineering team, without stifling innovation or creating unnecessary friction. Learn how to empower developers to build secure software by design, and discover the tools and strategies that are shaping the future of DevSecOps.

Show Notes

Transcript

Andrew Zigler: 0:09

Welcome to Dev Interrupted. I'm your host, Andrew Ziegler.

Ben Lloyd Pearson: 0:13

And I'm your host, Ben Lloyd Pearson.

Andrew Zigler: 0:15

In today's news, we're talking about a few topics. Top of mind for me is Google's new AI search mode. Another one is a new buzzword I was reading about called Ax, and not like an ax that you hit someone with, but actually agent experience. there was another one, uh, AWS formed an internal agentic AI group. That was pretty cool, and I found some sweet, sweet, useless apps online. Ben, what do you want to talk about first?

Ben Lloyd Pearson: 0:46

As much as I love useless apps, I think we should dive into the Google search.

Andrew Zigler: 0:52

Okay, so we've been talking about the search war between AI and Google on this podcast for a few weeks now, Google search has unveiled a new AI mode that lets you ask complex. Questions with many different parts directly into the search bar. before, what might have been five or 10, or maybe even many more individual queries by you, as you try to understand a question or seek something out online, you can now explore these search results in a more natural conversation like way with Google.

Ben Lloyd Pearson: 1:26

Yeah, it seems that Google search lives to see another day. you know, it's, kind of extraordinary to think about how much our typical research process has changed. I. In such a short amount of time, like the way we used to acquire information just feels like a, a distant dream at this point. But, you know, and I think this is also a great example of how Disruption from AI doesn't necessarily mean death. You know, a lot of people are concerned about whose jobs are gonna get taken or, or what sort of industries it's gonna wipe out. And I think the reality is more like AI is going to disrupt us, but it's probably not gonna wipe out like swaths of professional jobs overnight. Like there's a lot of room to just adapt and leverage all of the benefits that these tools are bringing us.

Andrew Zigler: 2:12

What I see here is Google taking another bite back from AI and the ongoing search wars, earning back a little bit of that market share. I think we've all had our mixed experiences going to Google and getting AI results. I. Answers, uh, in, in response to your queries. And so once that further towards a better version of that goal, is really great. And like what you were saying, I think that, you know, there's more disruptions always on the horizon. They come from everywhere, but ultimately what this disruption means is a higher demand. For more unique experiences.'cause now you can go to Google and ask that really in depth, maybe even personal question and crawl across a lot of real time resources in a way that maybe before you were using an LLM for anyways. It's going to create a more personalized experience for everyone online, which I think is gonna just increase the demand for more jobs and more professionals and more technologists that come in and create these personalized experiences for people with the technology.

Ben Lloyd Pearson: 3:07

Yeah, and this, this ties nicely into the article that I wanted to bring up today. and that's this concept of a agent experience. So, maybe this is disrupting Dev X, or at least that's the point of this article. that I read this week, and the whole point is that we have to start focusing on agent experience, which is defined as the holistic experience that AI agents have as the user of a product or platform. So, think about how everything is sort of built for humans as the end user, like the web for example. Most websites are built for a human to visually analyze them and extract information from them. And if you're a bot, that's just like scraping text off of a website. The web actually looks entirely different to you. Uh, and in that same way, AI agents are the same. the patterns that work for humans aren't always gonna work for these agents, at least not in an optimal way. this article highlights two ways that agents are getting integrated into products. the first is vertical, so that's like what we were talking about with the Google search example. and that's where you have tighter integrations of internal AI into your product. in addition to Google search, adding these new AI capabilities, it's like Google also adds like Gemini buttons to Google Docs, for example. and then the other side of that is horizontal, so making it easier for external agents to interact with your platform. So think about like, bring more, select external models into, to a platform to leverage capabilities that it has.

Andrew Zigler: 4:39

Thinking about the agent experience for the first time is, very eyeopening. it kind of shows us the way that we're gonna build technology to be consumed in the future who are its future consumers. this kind of news also reminds me of the recent, model, context, protocol, and all of the development. Around that. And, and if you're not familiar, I talked with a guest this week, about MCP, um, and he told me all about how companies are turning their products and tools, for LLMs into things they can use. I'm excited to bring 'em onto the podcast in the future, but it really got me thinking, about. This entire experience, because this is exactly what he described, creating the schema of the tooling for you to plug it into cursor for you to plug it into your at home chat box, for you to put it into any kind of in product or, personalized experience. And so, the idea that they have an experience too, and there's things that they need, it flips the entire script on how we might build, our actual workflows to be used. And ultimately I think it's time to think maybe a little bit about that agent experience alongside your developer. One, uh, when you're creating resources or otherwise developing your technology, think about who we might be using it.

Ben Lloyd Pearson: 5:48

Yeah. So what, what's, uh, what's the next story you wanna talk about, Andrew?

Andrew Zigler: 5:52

Well, we gotta talk about the useless apps at this point,

Ben Lloyd Pearson: 5:55

Oh, yeah. Yeah.

Andrew Zigler: 5:56

been talking about a lot of AI stuff. This one's a little more fun. Have you ever kind of wished that your phone kind of just like, wouldn't let you use it at all? Maybe Like at all. At all? Do you ever pick it up and you're just like, I wish I wasn't picking it up as much.

Ben Lloyd Pearson: 6:10

Yeah, at least twice a week.

Andrew Zigler: 6:13

At least twice a week. Right? And there's all sorts of ways you can fix that. Now, you can put privacy locks on your phone. You can put time locks on your phone. Here's a really fun one. someone built an app that doesn't let you open any kind of media app on your phone unless you literally verify that you've. Gone outside and touched grass. So you have to open the app and then walk outside and find a patch of grass and reach out with your camera, facing your hand and touch the grass. You can't actually use anything until you go outside. Right? Go touch some grass, like get some perspective. Cool off for a second.

Ben Lloyd Pearson: 6:45

Yeah, if you've been on Reddit any length of time, somebody has probably told you to go touch grass.

Andrew Zigler: 6:51

I love useless apps. This one though, for me, it's like I immediately thought like, oh, how could I, how could I break this? Could I use this app? And then like, could I print out a picture of grass and, oh, it's probably a little more sophisticated than that, but you know, like, I actually have a bonai tree. What if I like faked some perspective? Right. And it thought I was outside. So that was immediately where my mind went with it.

Ben Lloyd Pearson: 7:11

Well, you're illustrating that, for it to work for me, it would have to be go touch snow, like, uh,

Andrew Zigler: 7:17

Oh, very true.

Ben Lloyd Pearson: 7:18

under a foot of snow.

Andrew Zigler: 7:19

I didn't even think of that. What if you're somewhere where there is no grass? I think maybe there maybe needs to be regional variants of this app.

Ben Lloyd Pearson: 7:26

Yeah. But, you know, this is such, this is, I think, such an emblematic, representation of how AI is gonna change software development. Let's set aside, all these engineering teams who build these robust platforms and need complex orchestration of AI agents to, just think about the everyday person who doesn't know how to do all of that stuff. one of the common reframes about generative AI is like, it's pretty good at building hobby apps, prototypes, proof of concepts, et cetera. whether I wanted to like create a version of this that works for me where I can go outside and touch snow, or if it's somebody who's never built software before, but has a great idea. For a simple app, it's empowering like all different types of people to create new things that they weren't capable of doing just a couple of years ago.

Andrew Zigler: 8:14

I completely agree. We're heading for a time where anyone can spin up very personalized software on a whim that can serve a very specialized or personal niche for them. Maybe it doesn't have to work well, doesn't have to scale literally at all. We're talking about one user, maybe even for a temporary amount of time, and I think we're gonna see a proliferation of these almost disposable like. Apps and technologies that pop up just to explore an idea or to build an on something, or in this case, maybe even to make a bit of art. Because I think that AI will ultimately lower the barrier for people that are not in technology to build and create things. we saw the same exact thing happen with video games, which was a very highly technical field, that then slowly became democratized by tooling, to become more accessible to artists. And now you see video games as an artistic medium. I think you're gonna see the same thing happen with software and with applications. because when you unleash it on a world of people who don't think like technologists, they're gonna build things in totally different ways, and we're not even ready for it yet.

Ben Lloyd Pearson: 9:16

Yeah, I think there's a bold prediction you're hiding there that I maybe you should own, like will AI normalize software as art? I think Andrew Ziegler says yes.

Andrew Zigler: 9:25

I absolutely think so. I'm excited to see what kind of things come out of the future.

Ben Lloyd Pearson: 9:30

Yeah. Well, I, I love how every topic that we talk about, even if it's not about AI, always comes back to AI. I think we've got one more AI story, Andrew. So why don't you cue that one up

Andrew Zigler: 9:40

Oh, okay. Great. One more AI boomerang for you. So, this one is, you know, Amazon AWS, everyone knows it. They formed a group internally that's focused on agentic ai, this is yet another large company we've been talking about on the podcast, taking yet another, big move to form, their internal teams to put together their greatest minds. To focus on how AI can transform them internally. and it goes back to, Conway's Law even, uh, a basic principle in software development that you ship your teams, right? Your software products reflect your team structure. And so if AI is important to you and it needs to be everywhere and it needs to be parts of your product, then it's important that you. Put together a task force, put resources behind a diverse group of representations of users in your company to figure out how to make this work and to get it in place. and you know, we saw this with Goldman Sachs, we saw this with Microsoft. so here is yet another one from AWS.

Ben Lloyd Pearson: 10:39

Yeah. humans managing AI agents will eat the world. You know, we've heard software ate the world. We've heard open source ate software. I, I think humans managing AI agents really is gonna be the future of this all. And I actually wanna make, just a completely shameless plug for, the Engineering Intelligence Newsletter over on LinkedIn. This is a, new newsletter from LinearB B that I've been contributing to. And in last week's edition, we shared a whole bunch of practical advice for how you can start adopting agentic AI into your software delivery lifecycle. So we have all these giants, you know, Amazon, Microsoft, Google Meta, everyone out there is adopting agentic AI and there's no reason that everyone that's listening to this right now shouldn't also be doing that. So I'm definitely gonna recommend everyone go follow LinearB, be on LinkedIn, subscribe to the Engineering Intelligence Newsletter. A ton of great news happening over there. Yeah, so Andrew, tell us who do we have on the show this week?

Andrew Zigler: 11:37

Oh, I'm really excited for this guest. So after the break, we're bringing David Mytton onto the pod. He's the CEO of Arcjet a security SDK for developers, and he's the founder of console.dev, which is a weekly newsletter where he focuses on a curated selection of interesting developer tools. David has a really discerning eye for what's good and what's not, what you should use and what you should try. He tries everything so you don't have to, and his experiences go really far. He has a finger right on what makes for a delightful developer experience, and I'm really excited to share our conversation with y'all. So stick around for the discussion where we talk about the realities of shift to left in security and the status quo of DevSecOps. Are you struggling to explain developer experience to non-technical leadership? Join LinearB's upcoming workshop and learn how to translate Dev X into language the business cares about. We'll show you how to present data on developer productivity, AI performance, and engineering health in ways that drive alignment and investment. Plus, you'll get an early access to our CTO Board deck template, making it easy to connect engineering metrics to outcomes like faster time to market and cost savings. The link to sign up is in the show notes. We hope to see there. I'm delighted to be joined by David Mytton, CEO at ArcJet and the founder of the Console. dev weekly newsletter. David, thank you for joining us today.

David Mytton: 13:07

Thanks a lot, Andrew.

Andrew Zigler: 13:08

You're a seasoned developer and founder and an angel investor as well, and We brought you on the show to chat about your approach to shift left within DevSecOps, which is something that you describe as largely buzzwords that haven't achieved their intended impact. And I really want to start there. So perhaps you can set the stage for us about why the shift left approach is applied to DevSecOps and what are some of the issues with it.

David Mytton: 13:35

It's really modeled off the very successful DevOps philosophy. And this takes us quite a long way back in history to when development was entirely about just writing code. And then you would hand it over to somebody else to deal with, In the very old days, that was physically shipping it. And so ops wasn't really a thing. but as services became more popular and, we needed to access things that were constantly updated, then it became, well, how do we run the software that we write? And traditionally you would have sysadmins and then later on that broadened out into different types of operations. But essentially it was a separate team that would deal with deploying and updating and making sure the infrastructure, the servers and the network was all running smoothly. As we moved to more of a cloud environment, then the physical side of it kind of disappeared, obviously still there, but for most people using the cloud abstracts, the compute, the storage, the data center, those kinds of things. And so as more software became involved in the operation side of things, like software defined networking, the actual virtualization of the system, it became a question of, how do we run this entire environment? And separating. Code that, really if you write code, what's the point if it's never run and whose responsibility is that? And then when something breaks, who should fix it? Operations teams didn't write the code. So why should they fix it? Or even could they fix it? Cause they don't know what's going on. And so this idea of DevOps just meant that developers took more responsibility for running the code that they wrote. And that became testing it and setting up CI, CD, as you got closer to production, deploying it, and then when it's actually running, dealing with monitoring and any incidents that happen. And that was pretty successful because it's quite. Easy for developers to understand why do they need to run their software? It's because what's, why are they writing in the first place? It's to deliver a service or to provide something to users. And so it's, it brings them a lot closer to that and you can replicate the production environment very closely on your laptop or in a staging environment. And so it's a lot easier to understand how to actually set it up and test it and make sure it's all working. And so DevSecOps, kind of merged into DevSecOps with this idea of, well, if we're getting developers to run the code that they write and take responsibility for it, then why can't they also do the same thing with security and take what the security team had typically done and let developers take responsibility for it? That was kind of the idea. Does that make sense?

Andrew Zigler: 16:19

It does, it does make sense. And so based upon the story you've told, it's really a growing amount of abstraction and complexity away from being maybe a more physical based computing solution into something that's abstracted in the cloud. And each step on that process, the developers get closer and closer to where things are released and things are secured and tested. Is that kind of what you're describing?

David Mytton: 16:44

Yeah, that's right. it's giving developers more responsibility so that they can understand everything that goes into. Actually running the code that they write. And so this then led into the specialism of site reliability engineering, which was popularized by Google and a few other, the larger tech companies. Because if developers are running their software, what else is there to do? Well, there's quite a lot to do. And there's a lot of specialist knowledge involved with large scale systems, making sure they're reliable and that when developers are building, they don't have to understand the full details and can build on top of a platform. That's kind of what you're doing with the cloud, where you're building on top of Amazon or Google or Microsoft's platform. They abstract out all the data center. And as a user, you just have to think about the particular primitive that you're using, the compute or the database or whatever service it is. But underlying that there's still a lot of operations work. It's just, you're paying someone else to do it. And so the idea with DevOps and SRE was that developers can do a lot of things, but there are still things that are probably not a good use of their time. Or there are specialists that can help build out a platform for larger teams. And so SRE became this specialism where they would operate the internals and the underlying systems. And then they would give tools to the developers to allow them to autonomously deploy onto, onto those systems. And again, that's worked really well. SRE come in and maybe advise on the design of a system, and then the developers will use the platform and the tools that they've created. And ideally SRE are not involved in particular projects, but they are responsible for the whole system. The system as a whole. the idea with DevSecOps is. Basically the same thing. Developers should take more responsibility for the security of their software. And then when there's a problem, inevitably they're going to have to fix it. But perhaps you could have a specialist security team who would give them tools to make that security job a lot easier. unfortunately it hasn't really worked out that way.

Andrew Zigler: 18:43

And why hasn't it worked out that way in execution?

David Mytton: 18:47

I think it's down to the incentives. So for a software developer, their job is to build things either because they're building an internal system and they have internal customers who need something, or they're externally providing a service, product, service to, to users, and so you've got actually customers paying you to build things. So that's the developer's job, build something new, fix bugs, and deliver value to customers, essentially just building. The incentives for security people are very different. They are either there to break something because they're doing pen testing or looking for vulnerabilities, or they're there to mitigate risk, which generally means stopping things from happening. And that is basically the opposite of what developers are responsible for. So even before we get into any of the details of specific things that might've failed with DevSecOps, the incentives just don't align between the security team and the development team. They have to work together in many cases. Which can be a challenge. there's often a lot of friction there, but this fundamental mismatch in incentives is the big problem because everything comes down to incentives really.

Andrew Zigler: 19:54

Right. So the devs are there to build stuff and to make things and security there. And sometimes in the devs mind is there to stop things or to halt or slow things down. And then the security team only wants things that are secure, obviously, to go through. So you end up in this state of tension, where both of their incentives of what they want out of the engineering process are different. Is that because of how DevSecOps has shifted the responsibilities more into developers, or is that a response to that tension that is felt between security and dev professionals?

David Mytton: 20:28

That was a response because ultimately all vulnerabilities come down to some error or problem in the software that someone has written, or the processes around it, and so you always have to involve developers in the security process. There's no getting away from that. But because the incentives were so different, developers are incentivized to build things. They're on deadlines. They've got to deliver customer value and security is often a distraction from that. Now it's a distraction for very good reasons. It's just, it's counter to that incentive or the real goals of developers. And I think that's why it's become such a challenge to encourage developers to think about security from the beginning. Because if you're in a startup, what's the point of securing something if you have no users? And as your company gets larger, you really only think about security either When you're required to, from a regulation perspective, or when your customers start asking you for details about your system, and that's when the incentives start to align again, because there's a customer asking you for something, so you should implement it. Otherwise it's really down to the developer to put the best efforts in or to do things because they think it's the right thing to do. And we know from human behavior that probably people try and do that, but maybe you don't do that all the time.

Andrew Zigler: 21:48

Exactly. And so if this state of tension is just innate to those roles, what can developers do in security professionals as well to bridge that gap and maybe foster that more security conscious culture that allows them to exist in more harmony?

David Mytton: 22:04

I think this is the real challenge and it's, developers are not thinking about security problems and, This has given them the reputation of not caring about security. And I think that was actually just, it, it's the outcome. It's what happens as a result of that thinking, but it's not a deliberate thing. in most cases, it's just, some cases it's a lack of knowledge. Um, but that's not an excuse really. companies run education programs and training and it's all really boring. And so that's the, that's one of the challenges. but it's really about. Developers are trying to solve a problem. They're not thinking about security because security generally doesn't help them solve that problem. And so this means that they might be dealing with API abuse or signup form spam or fraudulent credit card transactions. And those are problems that the developer thinks about. Now from a security mindset, the solution might be, well, we'll buy a bot detection product or we'll build rate limiting or, use one of the credit card fraud services and those are security solutions. But the developer's thinking, in a completely different mindset. They're not thinking in terms of a security product they should buy or a particular, acronym that they need to understand and implement in their system. They're thinking. Loads of people are using my API and it's costing me loads of money. or we're getting loads of signups, and they're all spam and they're not, um, they're not helping our service or we're hurting our, um, reputation with our credit card processor because we're getting loads of chargebacks. Those are the problems that developers trying to solve and security needs to provide the security team. The security industry needs to provide solutions to developer problems, not try and sell them another product.

Andrew Zigler: 23:45

Right. So this brings us back to the core of our topic of shifting left and applying that to DevSecOps. And some of what you're touching on now is I think really salient here about part of that is putting the right kind of security tools into developers hands earlier in the process. It makes it more part of their development process. journey as they're creating the actual software, and it allows them to have earlier conversations with security about what they're building as well. And so what are some emerging trends or tools from your perspective that maybe are delivering on this or making steps to bringing this more into harmony?

David Mytton: 24:22

The buzzword right now is Secure by Design, and this has been pushed in particular by governments. the US government has a lot of effort going behind that, and the EU has recently passed some laws around defaults. Secure by Design is a great idea, and the challenge is the actual implementation. What does it mean? You know, on the consumer side of things, it means things like when you buy a new Wi Fi router, it doesn't have a default password, and it can be updated for the next 5 to 10 years, so that when there are vulnerabilities discovered, they can be updated automatically. That's quite easy to build in security by design, or at least you know how to do it. It's, you can put on a checklist. It's a little more challenging to do that with the underlying software engineering approach. And how do we encourage developers to build security into their products when the incentive isn't there? Now, one answer is just change incentives by regulating and force developers to think about this. And there's a question around liability and, how the industry works when there's a big vulnerability found, whose responsibility is it? Is there a monetary value attached to that? that can be a good way to change behavior. but as soon as you get regulation involved, then, It becomes, well, who's got the most money to do the lobbying? How do you balance the different requirements? And then regulation increases barriers to entry, which has been the great thing about software. Like you literally need no qualifications. You can just write code and you can put something online and people can start using it, which is great for innovation, perhaps not so good for safety and security. so I think that's the big challenge around. Security by design in terms of how you actually implement it and enforce it. And ideally we don't have the government forcing people to do things. Maybe that's where it ends up if it doesn't change, but I think the fundamental is around behavior and how developers think about this. And it's very difficult to change human behavior. And so you have to change the underlying system so that developers, people, whatever the system is that you're thinking about, the user of that system gets the benefit for free. So they don't have to think about it. So by using that particular system, they automatically get the benefits because the operator of the system or the designer of that system has made that change on your behalf. That is the responsibility then of the developers of infrastructure and the tooling to encourage developers to do this or to use these systems that give them security by default.

Andrew Zigler: 26:56

So if there's, if there exists a class of tools that allow developers to implement security practices earlier in their process, I'm curious from your perspective, because you have so much experience reviewing hundreds of tools for your newsletter, console. dev, which we're going to talk about a little bit more in just a moment. And when you approach this idea to evaluating these tools or thinking about how can we put. Better tools in developers hands earlier in the process. What are some of the things that you look for as someone who's looked at a lot of tools, and it has become very discerning for, for what we put in developers hands?

David Mytton: 27:33

Yeah, so for console. dev I've reviewed, um, probably 300 or so every year for the last four or five years now. So I've played a lot with a lot of tools we put into, uh, two or three every week. so those are just the ones that actually get through. And the first thing I think about is. would I use this and having been in the industry for almost 20 years, I kind of have a, it's my opinion on this. But then there are some specific things. So the most common problem that I see with a DevTool is the documentation has some flaws. Often the quick start guide is fine and it works. Sometimes it doesn't. And then that's essentially it's off the list straight away. Cause if the quick start doesn't work, then no one's gonna be able to use it. But the common failure scenario is that you, you do the quick start, the tool does what is advertised. It's got a load of other features, but you have no idea how to use them because none of them are documented. That's a real problem. Developers don't necessarily like writing documentation. I spend a lot of time at ArcJet writing documentation for our products, for our security product, just because I know that that is the, like whenever I'm trying a new dev tool, the first thing I'll do is go to the docs. And so that needs to be part of the product. we consider the docs part of the product. the other failure scenario is. Once you've done that quick start and it's working, how easy is it to adapt the product to the complexities of whatever it is you're building? This is a failure of breaking out of the happy path. And a lack of flexibility in the design of the product or a lack of documentation explaining how to use all of the options and the configurations, because most of it, like you see all these templates for like a chat app or how to implement authentication, really straightforward. Those are important things. And building app is, is really fun, but that's not what most apps are and you've got like all sorts of different requirements that customers are asking for, and you're building all these different variants and that's often where DevTools break is because they don't work in these various scenarios.

Andrew Zigler: 29:37

That makes sense. you're touching on something that, um, is very close to my heart, especially when you talk about documentation and evaluating new tools. As a developer advocate, I completely understand the mindset of when you go to a new tool and you're trying it out For the first time, there's a few gut checks that you do. You go to the repo and you see, when was the last commit? You go to the documentation and you look at the quick start guide. And is it a chat app? And if it is, does it work when you read it from top to bottom? so those types of approaches are, I think it's, um, very delightful to find that they're universal when other people evaluate software as well. And so when you go down this path of evaluating software, and you look at the docs and it's about putting. Great examples in developers hands so they can envision the tool, and it's about the tool also being flexible for what they're looking to implement. Do you see these tools oftentimes as starting grounds, or do you see them as holistic solutions? Like, are they there to inspire or maybe guide a path for the developer to take? Start with this tool and then maybe kind of build it into something else. Or do you see it as like a one stop shop? Developers need to be looking for like that Swiss army knife.

David Mytton: 30:48

This is often the distinction between a really great open source tool that someone's built and has become popular and a product from a company. And the open source tools, typically they solve a problem. The good ones, they solve a particular problem really, really well. And they might have some side features here and there, but they're very focused on the specific thing. And that's great for low level infrastructure, dev tools, terminal interfaces, those kinds of things. You don't necessarily want that to become a huge, massive monolithic project with all sorts of different features here and there. And there is this concept of software being done. And I think very few developers think about products like that. certainly startups and businesses, can't think about that, I suppose, because they're always having to add the next thing for growth, but certainly for open source tools, you can. You should consider, well, is this done? And there are very few that I see that take that approach, but the ones that do, to your point about, well, when was it last updated, perhaps they haven't made any major changes to the code recently, other than like putting independency updates and just making sure it continues to work on the platforms that support it, when I see a tool like that. And then that is very interesting because it's well maintained, it's within the scope of the original idea. It's basically done and it's just in maintenance mode. I think that's really rare these days to find software that just like, yeah, it's done and we'll maintain it, fix a few bugs, keep dependencies up to date. But this, it is what it is.

Andrew Zigler: 32:18

That's absolutely right. I think, um, done software, finished products is very mythical. You know, you don't really see those. And when you do, you wonder if that is really what they are. Are they really done? And so, this really brings me to my next topic of console. dev just in general. I'd love to learn a little more about, how this newsletter came to be. I myself have checked out console. dev. You make great tool recommendations. what I personally, love about, the newsletter, and I'm sure our readers and listeners will as well, is that it's very straightforward, but opinionated in a way where I'm like, I could get behind, this tool, yes or no. I can go through the list and understand if it's going to work for me. so I'm kind of curious, how did you cultivate that voice? And what inspired you to go down this road of, uh, sharing developer tools more broadly?

David Mytton: 33:05

Well, it's really just my opinion on things condensed into 300 characters of what I like, 300 characters of what I don't like, that is the unique bit that I still haven't seen anywhere else, really. And it's only on a newsletter format. there's lots of really great newsletters and often they're just loads of links and they're just the interesting things that happened in the period that the newsletter is sent and that's great. It's a good format, but it's kind of passive reading. You might not read it every week. You probably might not discover anything interesting every week. and it's just general interest. When I was thinking, before I started console, I just wanted to know, how do I stay up to date with all the things that are happening in tech and software, in the cloud and services, open source, and whilst you find those things on Reddit and Hacker News and Twitter and BlueSky and all their various social networks, there was nowhere that was focused purely on developer tools and, uh, And then there was no recommendation that you might see someone tweet something. And then next thing they'll be tweeting, something else or a photo. And there's no this consistent discovery engine. And so I thought, well. I could do that because I like playing with DevTools. I really love technology. and I have an opinion on what works and where, because I've run services that scale, I've built software in different languages, I don't know every language and every tech stack, but I can probably assess something in a way that another tech stack doesn't. Experience developer would do and the goal with the newsletter is to give you something at least once a month that you actually think, yes, this is great. I could use this personally or at work. Maybe you'll find more tools, but at least once a month is the goal. I subscribed to several thousand blogs. I've got a feed reader, I've got filters. I've got automated scripts that just highlight all the new beta and alpha releases of tools. And then the reviews of tools are. It's just things I come across and it might be a brand new thing that was just released, or it could be something that's been out for 10 years and I've just discovered it or I've been using it recently, or there's been a new version that I want to have a look at. So it's all sorts of different things and there's no particular theme other than DevTools that are good.

Andrew Zigler: 35:17

And I'm curious too, when, when you have reception from folks who read your newsletter and they say, Oh, I loved this, or I didn't like that. What are some of the things that, or challenges or pain points even, that you think developers look for very often in your content that, for your tools to solve for them?

David Mytton: 35:35

the key thing with the why I don't like section is it's less about, this sucks about the tool. It's more, these are the limitations of the tool that you're going to discover. If you try it out in five or 10 minutes, you'll discover these things. and because I only include things I think are actually worth your time. It's never, I don't feature tools I think are bad, but there's nothing that is a hundred percent good. And so it's really looking at. Well, what do developers use on a regular basis? They're on their laptop, their system. So terminal utilities, ID integrations, things that help them write software. And then there's cloud utilities, products as well that can be commercial products, things you have to pay for, but things that are going to help developers with their job. Um, it might be a paid editor. It might be a cloud service that is part of their core infrastructure. It's really anything that a developer might want to use. so the most popular things that show up are editors. I think most everyone probably uses VS code. There's been an explosion recently of lots of really interesting new editors, whether it's kind of VS code. Forks that have added loads of cool AI stuff, or it's brand new editors, like Zed, which is implemented in Rust from scratch. there's like Mac specific ones like Nova, there's Vim of course, and all the different plugins that you might have. there's a lot of really interesting innovation happening just on the editor side of things, and that tends to be the most popular category.

Andrew Zigler: 36:56

Oh, completely. I think the developers get very specific about things, have dot files and they can configure in that kind of way, especially when you get really close to the IDE and you're touching on something that I've noticed in the last year or two years is that you're getting these tool categories that have typically been kind of, you know, So, Quote unquote solved for, in many ways, suddenly getting disrupted, obviously, by AI and transformation in general within the tech industry. And I'm curious, what are some of the, biggest pitfalls that you see new innovative products like fall into when they try to reinvent a category, or re approach something that developers in their minds, you know, think has solved?

David Mytton: 37:32

The most interesting one, I think, is probably the terminal. I thought, the terminal has been installed, that is done. You're either using iTerm, or you're using the one that's built into macOS. Windows obviously has one as well, and Linux has a couple. Alacrity and, uh, Um, Kitty and a few others like that. but I essentially thought this is solved. It's open source. There's never a business here. And we've seen some interesting startups appear, which I still have some skepticism about the business model around selling a terminal. but I think there's going to be lots of, there is a lot of work happening there. I think AI makes terminals particularly. More interesting, but it's because of having to remember all the commands. You're going to remember a lot of them. very few people actually read the manual, and don't know the man command and it's sometimes not even that helpful. So I think that's where AI can make a lot of difference. people have a lot of opinions on their terminal editor, the terminal emulator that they're using. and I think it's a very tricky, sector to get into, almost as difficult as code editors because developers, my experience, don't pay for anything ever except for their editor, um, by default. But the company may buy things for them and that's where the additional tools come in. But as far as the developer individual paying for stuff, it's very, very unusual, apart from the editor itself. so I think there's an opportunity there and we've seen a few startups and adding and innovating on that as well.

Andrew Zigler: 38:55

you touch on something very funny to me, which is, you know, developers don't like to pay for products. And maybe this is something that it gets to the root of the differences between their incentives with the business and with security even as well, because developers, just want to build, they want to make, and they want to create and their tools that they use, they need to delight them, they need to have The ability to do the things that they need to. They're, they're less interested in like, Oh, what's the pricing model? What's the subscription tiers? What are the things that I need for the enterprise level in order to deliver XYZ? So do you find that when developer tools are evaluated, it's almost like they're evaluated in two different ways. You have the business evaluating it for the business needs. You have the developers evaluating it for the developer needs. How often are those in conflict to adjust like how developers and security are?

David Mytton: 39:44

This has been an ongoing challenge, I think, linked to the sustainability of open source and how very popular and important projects have historically been developed by one or two people and not full time, and there's been huge burden on them. And then when you charge developers for things that you go on, on Reddit and people are complaining a lot about, well, I don't want to pay 10 a month for this Critical service that I'm using in my side project, or they write up about how they saved a hundred dollars a month by spending four weeks building a custom system. And so there's this real disconnect between the value that developers get and the value that they think they're getting and how they attach a dollar amount to that, and I think that's where, when you're building a DevTools company, the developer might be the user, but the developer is very rarely the buyer and it's very difficult to sell the value to an individual developer. This has become a thing, certainly in the last five or 10 years as DevTools startups have become a real thing because developers have a lot of influence. And that's how you can really sell into organizations is when developers love a product, then they're going to recommend it. They may not buy it themselves, but they will recommend it. And then they can get access to the budget or convince the budget holder. But it's a real. Tension because people don't like it when services shut down or when they get acquired and often it's because they couldn't figure out the sustainable business model behind whatever it is that they're building.

Andrew Zigler: 41:11

If the developer is the champion and not the buyer for the software, or they're the advocate for its usage, what are some key principles or best practices that you would give to like a DevToolings company or new kind of tool that's trying to strike a balance between appealing to that champion and appealing to that buyer?

David Mytton: 41:33

I think you have to take it in stages. So you have to work for the user first, because if you don't have the user, then you either go out of business or you have to take the approach of forcing people to use things. And we've all been forced to use horrible systems, probably some kind of expense management system that we hate. And that's because the user isn't the buyer. the buyer is some other department within the company That deals with a completely different part of the product. and so the user experience is terrible. And so that's where we're doing some, what I think is a, an unusual approach with security is because most security products target security people. And we'll have to go there at some point. We'll have to build features for security people, but most security products are not built for developers. And that's what we're trying to do with ArcJet. And then later we'll go to, the security organization and the infrastructure team and the CTO and show them the value that we're adding. And hopefully it'll be obvious, but the developer is going to be adopting the product. The big challenge is that you can't just put an ad in front of a developer and expect them to sign up as a result of it. That does happen, but normally they will have seen the products many times before in different forms, they might've seen a YouTube video, they might've heard someone talk about it on a podcast. Then they read a comment somewhere. And then when they see an ad or maybe a blog post or something completely different, they click through and that's when they take the action. And this makes marketing to developers very, very difficult because they are very, very astute and they know when they're being marketed to, But it's also very expensive because you have to cover every single channel and you don't know which ones are working. And I think this is what has made the rise of influencers, particularly on YouTube, become an interesting phenomenon over the last few years, because it is social proof from someone that you trust. And although it is still marketing and there are sponsors and, and they take money for different things and they're, they're usually very upfront about that. it is expensive and developers are probably one of the hardest groups to sell to.

Andrew Zigler: 43:42

Absolutely. Developers are very astute. I say as someone myself who's evaluated many DevTools and used every trick possible to escape around those lead gen forms or to otherwise get my hands on what I'm looking for. Or if I'm evaluating doing so almost like I'm behind enemy lines and I'm incognito. I don't want them to know I'm here looking at it. I don't want the emails. I don't want the blog posts. I don't want my LinkedIn feed to change, right? so that's kind of an interesting dynamic, of course, when developers do go in and evaluate their tools, which goes back to what you you mentioned earlier about being a really strong thing you must do of having great documentation, because that is where they go to find out those things when they don't want to be marketed to, they want to go read the documentation, which can be its own form of marketing, just like how the influencers on YouTube can be. but the difference here. is that you're allowing the developers to evaluate and draw their own conclusions. And I, I think that that's, really the heart of what helps developers evaluate the best tools is when they're evaluating it for themselves on their own standards and basis, not on someone else's rubric. Do you agree?

David Mytton: 44:46

that's, yeah, that's exactly right. And what you just described is how not to do developer marketing, like force them through a form or just speak to someone, like you'd look at any of the good dev tools, any product that developers are involved and you can go on the website and firstly, you'll go and scan the docs and the docs should give you a very good sense of how to set it up and the value that you're going to get very quickly. Okay. And then you should just be able to try the product. You have to be able to sign up, install it, run it locally, probably, or in whichever environment it's going to be and actually test it out and 99 percent of the time, you're going to log in with GitHub, probably, maybe Google, but certainly GitHub and you're going to get a Gmail account. And so that's how companies should do it. They should make it really easy. Like a one click sign in with GitHub, don't send any emails, except maybe one with a link to the docs and some interesting resources, and then let developers test it out. And you can instrument your, your products. So you can see when people have signed up, when they've activated, when they're actually starting to get value. And there are different points when you might want to reach out and say, like, if there's a long gap between sign up and nothing has happened, maybe you might want to say, did you have a problem? Did it not meet your requirements? But more likely it's once they've installed it and you know that they have seen some value, That's the point, to send a message and just say, how did you get on? Did you check this doc? This really helps, and particularly if you know where those users are coming from. Like for , we integrate into Netlify fly.io, al, and when we know you've come through their marketplace, we'll send you a couple of links just to how to connect it up or to set it up in a certain way and you can start making these little. Tweaks and adjustments just to help the developer figure it out because they're going to figure it out themselves. And if they don't, then most likely they're going to open a GitHub issue or come on discord, or they're just going to disappear. And so just making all these channels available, you don't need to force developers, you don't need to trick them and just remove all the friction. that, that's where I see a lot of success.

Andrew Zigler: 46:46

just focus on building something great that delights them and solves their problem. And they'll do the marketing for you.

David Mytton: 46:53

well that, yeah, that, that's 50 percent of it building the product. I think the other 50 percent is the awareness. And once they're, once they're into the product, then you make it really easy and they'll do the evaluation. so long as they have the right docs and the resources to do it. the tricky bit is just the awareness because. Just because there are so many tools, there are so many products trying to reach developers, so many open source things, there's open source variants of startups and commercial products. And just getting the mindshare of the developer is very, very difficult.

Andrew Zigler: 47:23

David, this has been an amazing conversation I'm curious, where can folks go to follow you and learn more about what you're doing and working on, whether it's ArcJet or otherwise? Where can they find you?

David Mytton: 47:34

So you can subscribe to console.dev on the website, free every Thursday. And if you're interested in what we're doing at ArcJet, then it's arcjet.com.

Andrew Zigler: 47:44

Great! We'll make sure to put the information in the show notes so our listeners can check it out. That's it for today's show. we hope that you enjoyed our conversation today with David Mytton. And if you're enjoying these insights from engineering leaders, please make sure you head over to the stack, uh, for weekly articles and deep dives on these kinds of episodes. And also don't forget to subscribe to our YouTube channel where we can put all of our favorite moments from this episode and other ones as well. The last thing I'd like to say is we always love to hear from our audience, so please leave a comment, rate us directly within your favorite podcast app, and let us know what you thought about this week's episode. David, thank you so much for joining us today.

David Mytton: 48:26

Thanks a lot, Andrew.