"If you wrote better comments, Copilot did a better job for you of coding, and so it kind of tricked you into writing better comments and better interface names and better documentation... Communication is key and it just gets more important with the advance of AI."
The single biggest predictor of success with AI isn't the model you choose, it's the DevOps culture you've already built.
Martin Woodward, VP of Developer Relations at GitHub - and the sixth person to ever use Copilot - joins us to explain why this surprising insight is key to the new era of autonomous coding agents. He traces the evolution of GitHub Copilot from a simple autocomplete to a powerful agent that opens its own pull requests, arguing that AI's true power is as a massive accelerant for the iterative loops high-performing teams have already perfected.
Martin explains that teams with strong guardrails for shipping quickly and safely are best equipped to leverage this AI revolution because they can trust the accelerated output. He also reveals how top teams use the key technique of custom instructions to guide Copilot toward writing the code of the future, not just mimicking the code of the past. This conversation uncovers how new agentic workflows are 'tricking' developers into improving their communication and documentation skills, providing a crucial look at the cultural foundations required to thrive in the AI-accelerated enterprise.
Show Notes
- Martin's GitHub Galaxy Keynote: Watch "The AI-Accelerated Enterprise”
- GitHub Copilot: Learn more about the tools and features
- GitHub Universe Conference: Look out for announcements for the upcoming conference
- Connect with Martin: Martin's Social Media Hub (Martin.Social)
- Connect with Erika on LinkedIn
Transcript
(Disclaimer: may contain unintentionally confusing, inaccurate and/or amusing transcription errors)
[00:00:00] Andrew Zigler: Today we're diving deep into the future of collaborative engineering with Martin Woodward and Martin's the VP of Developer relations at GitHub, he's actively shaping the tools that are defining tomorrow's engineer coming out of Microsoft builds co-pilot announcements and more. And Martin recently delivered the opening keynote about the AI accelerated.
[00:00:18] Andrew Zigler: Enterprise at GitHub Galaxy, his talk, which we'll include in the show notes. And I, I really enjoyed watching it from beginning to end. It dives into how AI is accelerating all steps of the development process and the things that teams can do today to shift into the new workflows of tomorrow. So Martin, I'm really excited to unpack this with you.
[00:00:37] Andrew Zigler: It's top of mind for so many of our listeners that I can't think of a better person to go to and ask for some of the insights about where this is going. So thanks for joining us today.
[00:00:45] Martin Woodward: You are welcome. Thanks for having me. It's always good to to chat with a fellow nerd about where we are and where we're going, you know?
[00:00:51] Andrew Zigler: I couldn't say it better myself. And so let's go into the shift that we're talking about, in development. And, before your, your recent, keynote at GitHub Galaxy, there was a major [00:01:00] announcement at Microsoft build around autonomous co-pilot agents that can now tackle issues, run commands, edit files, open prs, and, and tackle issues on GitHub and do it in the platform all on their own.
[00:01:11] Andrew Zigler: operating in a way that's more removed from developer workflows that we've seen in the past. what do you think this means for. The future of software engineering as we know it now.
[00:01:21] Martin Woodward: Yeah. it's worth saying. What we, did there was, it was the gi coding agent, you know, copilot coding agent and. What we're, what we are doing is we added, you know, we started off with copilot where it was sort of auto complete and you know, it, it, it, it gave you suggestions, but kind of the, it had 200 milliseconds to give you those suggestions.
[00:01:41] Martin Woodward: And so there's a limit into how much kind of. Thought it can put into that. How much reasoning can go into that before it gives you a suggestion back. Then we kind of added chat and then the chat. When you're doing a chat interaction, you can wait a few more seconds before you start to respond and it still feels natural.
[00:01:58] Martin Woodward: And then over time, we've moved to [00:02:00] where we're at now in say something like VS Code or Intel J or something where you, you have an agent mode, so you can go in and you can say to, you know, inside of your editor, Hey, can you help solve this problem for me? Or. You know, using a thing called MCP integrations, you can say, Hey, can you fix issue number 12 for me, for example?
[00:02:19] Martin Woodward: And it can go grab the context it needs. It can, automatically look at your codebase, see what type of codebase it is. Go do the thing that it thinks you're asking it to do, asking you appropriate questions along the way, and then, come back to you, you know, with like, Hey, this is it, and run tests and, you know, run linting and things and make sure it looks good based on what it sees.
[00:02:40] Martin Woodward: then you can iterate from there. So you had a very, very tight loop in terms of microseconds. Then with chat, you had a loop in terms of seconds. With agent mode, you have a loop in terms of, you know, seconds to minutes kind of thing is how long it would take.
[00:02:54] Martin Woodward: But you still, it's still very iterative with the coding agent. All that basically is, is [00:03:00] all, all the experience should feel like is, hey, rather than me sitting and watching vs code, while it codes for me,
[00:03:07] Andrew Zigler: Yeah.
[00:03:08] Martin Woodward: copilot is coding for me, I'm gonna background that. I'm gonna say, Hey. Co-pilot, create me a pull request that does this, or here's an issue I was gonna work on co-pilot, why didn't you make a start on it first for me please?
[00:03:19] Martin Woodward: And then, go work on it. And, you know, so we're doing it within the pull request in GitHub, and that's key because. you see a lot of these demos where people are like, you know, completely different. Ui. Take a picture of something today. You have a website. Isn't this great? You know,
[00:03:36] Andrew Zigler: Yeah.
[00:03:36] Martin Woodward: which is great. Demo.
[00:03:38] Martin Woodward: Sadly, that's not how the world works in terms of building stuff, typically. It's very iterative. Typ wait needs to be iterative actually, and that we found that. A key to success is actually speeding up the iteration loops. And that's how, 'cause that's how developers work, needs to be. But it needs to be iterative and it needs to be focused on, you know, small, discrete chunks of work that it can do and then come back to you and then [00:04:00] you can build onto the next thing.
[00:04:01] Martin Woodward: So we're hooking into that UI that you are already familiar with the code review, the pull request, doing some work there. And then the AI can come in. you can then review what work co-pilot's done. you can just take it as normal, and kind of, take it as the first pass of how it's worked and finish it off.
[00:04:19] Martin Woodward: Or you can go, actually, you know what, that's kind of close. Give it some more comments. Okay. It makes some more changes for you. Right. That looks good. It passes all my unit tests. It passes all my, performance tests. Everything is security tests. Everything looks good. Let's merge it and let's ship it.
[00:04:33] Martin Woodward: So that, that's kind of the thing that we showed at Build. And while it was, it was a massive, massive change in terms of how I. work. it was fairly incremental in terms of, the progression of steps of how we got there kind of thing. Does that make sense?
[00:04:48] Andrew Zigler: It does. It does. And when you made that announcement, you know, the, the world was really paying attention. I think everybody talked about it that week. It was definitely all I saw, anywhere and,
[00:04:58] Martin Woodward: Great.
[00:04:58] Andrew Zigler: and it made me, it made me immediately [00:05:00] think of, you know, how, how does GitHub see top engineering teams adapt to this new
[00:05:05] Martin Woodward: Yeah.
[00:05:06] Andrew Zigler: working?
[00:05:06] Andrew Zigler: These faster iterative loops require a whole new way of approaching building
[00:05:10] Andrew Zigler: things.
[00:05:11] Martin Woodward: One of the things I found, one of the things that concerns me, actually, and we'll talk about why it concerns me, but what, what I'm finding is the teams that do good DevOps, the teams that already have the guardrails in place, To make it so their teams of humans can build quickly and iterate and ship to production reliably, and learn from production reliably.
[00:05:31] Martin Woodward: Those teams are right now, the ones that are best equipped to take advantage of this AI revolution because they've got the safeguards in place. They've got all the tests, they've got all the, you know, the ability to push to production quickly and learn from production quickly. And so they're the ones that are able to take advantage of AI the most.
[00:05:50] Martin Woodward: and to your point, it's accelerating the teams. It's not replacing teams like we need more developers more than ever, but what it is doing is allowing developers to iterate more [00:06:00] quickly, which then allows us to put that flow of value to the end user more quickly and then learn from them. You know what I mean?
[00:06:07] Martin Woodward: And, so that's what we're seeing. the high performing teams doing and there's all sorts of things we can talk about along the way in terms of, you know, best practices, like, um, how you use custom instructions, how you, how you use these tools the most efficiently. but what I, what I hope and what I want us to try and make sure we, we do as well is.
[00:06:30] Martin Woodward: Allow teams that got left behind in DevOps to kind of short take a shortcut to be able to, like, what was stopping them from doing these things in the past. And usually it's one of two things. Um. From a technical side, it's, it's too much technical debt that they've not been able to pay down 'cause they're trying to show value to the customer.
[00:06:52] Martin Woodward: But the other one is cultural and the cultural one is much harder to address. Both in terms of internally in the team, but also the [00:07:00] trust you have with your stakeholders, the trust you have with management, that sort of thing. So there's a whole world of things sadly, AI can't fix and that one of those is, is culture of companies and teams.
[00:07:11] Andrew Zigler: Yeah, that's such a great point. That's something that we, we touch on a lot on Dev Interrupted. When we talk with folks building, you know, high performing teams and adopting new cutting edge software like this, is that end of the day, those things, they free you up to solve those deeply human problems that are very, they're next to impossible for AI to really make a dentin.
[00:07:30] Andrew Zigler: These are things around communication, collaboration, expectations, level setting, understanding where everyone's. Going. Right. And the AI
[00:07:37] Martin Woodward: Yeah.
[00:07:37] Andrew Zigler: necessarily do that work for you. It can do the steps between, but it requires that really good team understanding. And I like how you framed it as like those teams that had that DevOps culture that had that practice and, and had.
[00:07:50] Andrew Zigler: Built those processes and understood their bottlenecks and, and, and how they worked together as a team. They were already perfectly or really well equipped to [00:08:00] take advantage of all of this acceleration because you're just talking about adding in more iterative loops into a framework that was already agreed upon and understood.
[00:08:09] Andrew Zigler: Right. And, and so that's a really
[00:08:11] Martin Woodward: And we're set up to handle c continuous delivery at pace, you know, um, whereas I've worked places where, you know, you, there's nine months between ships kind of thing. And if you, if you're in that kind of world, is AI gonna help you as much there? If you can't, you know, it is. It's about how can we help this?
[00:08:30] Martin Woodward: This, you know, this, uh, speed of iteration really?
[00:08:33] Andrew Zigler: Yeah. And we're definitely gonna dive a little bit more into the, that DevOps parallel with taking
[00:08:38] Martin Woodward: Okay.
[00:08:39] Andrew Zigler: of, of AI and agents. And I'm curious too, just when we learn about how the agents would work and they can kind of work in the background, right? So you get
[00:08:48] Martin Woodward: Yeah.
[00:08:48] Andrew Zigler: almost like. there's higher level, there's higher order of magnitude work that you could possibly delegate and have done, but that
[00:08:56] Martin Woodward: Yeah.
[00:08:57] Andrew Zigler: a big shift in how teams like, are aligned in their [00:09:00] practices through DevOps, how they communicate and coordinate and assign
[00:09:03] Martin Woodward: Yeah.
[00:09:04] Andrew Zigler: are, are there like patterns that you see emerging in teams that are successful in the way that they co-work with an LLM that maybe stands out to you?
[00:09:13] Martin Woodward: Yeah. there's a few different things. One again, what's fascinating, and we, we saw this already with like the early versions of copilot, is that if you wrote better comments, copilot did a better job for you of coding and so it kind of tricked you into writing better comments and, and better interface names and better documentation, because the more context you gave the model, the more it was able to help kind of thing.
[00:09:37] Martin Woodward: We see a similar thing happening. I mean, it's early days again, you know, we announced. Like as we were recording sort of three weeks ago. but we'd obviously been dogfooding it for a while and dog feeding it with other teams for a while. what we're seeing is that, you spend a bit more time defining your issue, uh, because you find that you are writing it up less as an aid memoir for [00:10:00] yourself, but more of like a spec almost, you know, of like, how, what, what, and actually.
[00:10:07] Martin Woodward: Like this is, this is the whole freaking point of DevOps. Like, it, it's to have that conversation early with the customer of like, so what does this need to do? What happens in these cases? You know, all that stuff we've been doing since TDD and then through Agile and then through DevOps. It just kind of forces you to do that again because you, you wanna write instructions down in a way that not just you'll get and figure out, but that, uh, an AI can help figure out for you.
[00:10:32] Martin Woodward: Um. Or a human reviewing that code can figure out for you. So it encourages those kind of best practices. So that's the first thing that we see, which is kind of like accidental, but also cool and makes me think, oh, maybe this AI is like a good thing. Maybe this is, you know, this is, this is, this is definitely helping.
[00:10:48] Martin Woodward: The other one that we see is, custom instructions are the like the secret source that not enough people know about yet. And yet the, the, the teams that get the most out of copilot, that's what [00:11:00] they're using. it's basically a file you can have at different levels. You can have it like personally in your like local machine.
[00:11:07] Martin Woodward: you can have it in the repository or you can have it at the org level as well. And it, it's additional instructions that get passed to the prompt as part of every single indication. that means that there's a few things. One, you can define a personal style, for example. I make sure all my comments are in British English and that we, I add use in random places to words and things like this.
[00:11:30] Martin Woodward: Um, I don't use M dash in, like, don't generate comments with an m dashing, 'cause I never use M Dash. Like, why would I use M Dash in a comment? You know, and some of those sorts of like personal quirks, but more importantly at team level, I can specify, hey, when are we using. When we're doing a react component, let's stick this.
[00:11:47] Martin Woodward: Um, you know, we, we use a flagger as our internal feature flagging system, but say you're using something like LaunchDarkly or something like this, like wrap this within this feature flag automatically for me. 'cause that's our coding standard. [00:12:00] When you are logging, use this logging framework when you know you and with, with, examples.
[00:12:05] Martin Woodward: And by doing that, it allows the code that's generated by the LLM. To not just to reflect where you want to go with your code base, not the crap it finds in your existing code base. 'cause we all know our code bases are currently terrible and we want to make them better. You know, if, if an LLM's context is just what it knows from what it's been trained on, plus what it can detect from your code.
[00:12:29] Martin Woodward: You need to give it additional instructions to also go, and this is where we want to go. This is how we want the future to be with our code base so that it's continuously improving and continuously making things better. That's probably the main trick I see. Not enough people have kind of figured out yet, but Yeah.
[00:12:44] Andrew Zigler: There, there's two things now I wanna double click on. I'm gonna
[00:12:47] Martin Woodward: Okay.
[00:12:47] Andrew Zigler: in in reverse because, and, and the first one is the one you just talked about with the instructions. 'cause
[00:12:51] Martin Woodward: Yeah.
[00:12:51] Andrew Zigler: echo that as a practitioner, someone who's using these tools every day that, like the GitHub copilot instructions and, and being able to add that personal style [00:13:00] level to, to what you're prompting, just kind of baked in it, it
[00:13:03] Martin Woodward: Yeah.
[00:13:04] Andrew Zigler: it reduces like the iterative cycle of, of going
[00:13:07] Martin Woodward: Yep.
[00:13:07] Andrew Zigler: forth with the LLM, but. What it does actually is it forces you to understand what's important and what's not and then that gets to the main point of what you said and I think you said so well. Something that like has been bouncing around in my head a lot and you really like nailed it about how AI is kind of in a way tricking us into adopting and revisiting all of these like really
[00:13:30] Andrew Zigler: great baseline foundational best practices about how to make great software and, and, you know, doing things like TDD building a spec that we can all agree on that has the full idea of what's supposed to be accomplished. And really engineering for impact, right? For how folks are gonna use your software.
[00:13:46] Andrew Zigler: And LLMs, they, they require these guardrails and so we have to go back to like the basics and, and really write that stuff out and make sure we're all on that same shared page. So it makes everyone a better developer. And I, and it seems [00:14:00] like the key to going fast is, is to be really strongly aligned with what you're building and
[00:14:05] Martin Woodward: It's in communication. Again, that's why I think the teams that today do DevOps well are, are getting the most benefit because the better you can communicate, the better an an LLM can understand your intent and the better it can execute on it. But also the better your team can understand the intent, the better your customer can make sure you've understood their intent.
[00:14:29] Martin Woodward: So, um, communication is key and it just gets more important with the advance of ai, which again makes me very optimistic against some of the, you know, sort of, sort of some of the doom and gloom type scenarios and things like that. I'm like well, if it's tricking us into communicating better and tricking us into, you know, having better documentation and can actually help me create good documentation and can, Yeah. help me create good tests and can help my developers who don't speak English as a first language understand requirements that came in in English, and [00:15:00] they now want to understand them in their language.
[00:15:02] Martin Woodward: Like, these, this is all, this is all goodness, you know.
[00:15:05] Andrew Zigler: Right, exactly. And if it makes us all work better, then that's the direction that we should all go in. And that kind of
[00:15:10] Martin Woodward: Yeah.
[00:15:11] Andrew Zigler: our, the next topic that
[00:15:13] Martin Woodward: Yeah.
[00:15:13] Andrew Zigler: into with you about, like the skills that you see emerging for the developer of tomorrow. Like we're, we're moving into a, a really, a new skill set that developers have
[00:15:21] Martin Woodward: Yeah.
[00:15:22] Andrew Zigler: and this, this is where it gets really interesting. It's where there's so much opportunity and it's where like. The folks who, who are listening to us now, people who read Dev, read and listen to Dev Interrupted. This is what, you know, we're all on the cusp of figuring out together as, as developers, as engineering leaders of what are the things that I need to do for myself and for my team to make sure that we're upskilling and getting ahead and taking advantage of where this is all going.
[00:15:46] Andrew Zigler: And in your, in your keynote, you touched really well on like the evolution of LLMs from being like auto complete to being more multimodal. And then you're moving into an AI native environment and. what you've iterated here today is core to that, that those teams [00:16:00] with strong DevOps practices, they're the ones that are best positioned to take advantage of that. Can you like walk us through what that would look like in practice for a team and how leaders could consider maybe revisiting or taking DevOps more seriously so they can mature into this new era?
[00:16:18] Martin Woodward: Yeah. I think if a customer could come to you and explain exactly what it is that they want and how to best implement that in with technology, then an LLM could probably do that task entirely on its own kind of thing, but actually that's never, that's never the case for us. it's like, why I love technology so much and why I enjoy it every single day is because we're solving problems.
[00:16:49] Martin Woodward: Like it's that problem, creative, problem solving, being able to parse apart what a customer is saying and like, okay, so if I did this, this would [00:17:00] solve it in the minimum number of steps kind of thing. And this is what computers can do. And then also all those skills you have to build up over years of like how to break down.
[00:17:13] Martin Woodward: the bigger the problem is the more you need to break it down into small steps so that you can iterate as you go and you can increment and you can head make sure you're heading in the Right. direction. 'cause what you think is the answer at the, at the macro level definitely isn't the answer when you get down to that small level.
[00:17:28] Martin Woodward: So that skill that you already kind of have but is hard to build of taking down a big, multi sprint activity into, you know, individual day tasks kind of thing that you can get done. You need that skill even more in the world of ai because if you just give it a big, build me one of these things, it's gonna try and it'll spend a lot of tokens and it'll spend a lot of like time doing it, but it's more likely to not be exactly what you want.
[00:17:56] Martin Woodward: 'cause at any point along that random, you know, interest, [00:18:00] entropy path that it was doing to help interpolate what you wanted. It, it can make a choice and it might make the wrong choice, just like humans do. And so by being able to figure out, okay, how do I break this down? What components do I need? How am I gonna create those?
[00:18:14] Martin Woodward: And what you do is, you know. Again, you create issues for these, you create pull requests for them. But the, the, the major difference is that, you learn where to lean onto the AI for assistance. So, hey, well this sounds like a, like, you know, create to me a, a set of test cases. Well, let's let the AI make the.
[00:18:31] Martin Woodward: Skeleton for that. And then I can go see, go see what copilot missed and go see what other edge cases I need, and actually, ooh, that's now making me think about this problem even more. Let me go talk to the customer to see if this is an edge case or not, that I need to take into account. And it just allows you to be more creative and kind of.
[00:18:48] Martin Woodward: Frees you from the, the, the typing toil, the implementation toil and allows you to focus more on the kind of creative problem solving. So you then, as a, as a team, you have to make sure [00:19:00] you're building those skills. Building the, like, how do we break this down, into smaller chunks so that we can then have humans build some parts, you know, co-pilot help with other parts, collaboration between co-pilot and humans for other parts sort of thing.
[00:19:12] Andrew Zigler: Yeah, literally eliminate all the typing. I even now prompt my LLMs with voice to text and so
[00:19:19] Martin Woodward: Nice. That's the advantage of working from home. Yeah. Rather than in a cute
[00:19:23] Andrew Zigler: oh, I mean, I'd probably do it even when I was in an off. This, I, I'll be one of those people, but I, everyone can just see how they, they can just see how I blunder through my prompts in the open, you know, we can all learn together.
[00:19:34] Andrew Zigler: Which, which actually brings me to another point that I really was curious to know about at
[00:19:38] Martin Woodward: Hmm.
[00:19:38] Andrew Zigler: is how do y'all foster like a, a sense of continuous learning within your engineering team? 'cause this is like a total like, sea change event and y'all
[00:19:46] Martin Woodward: Yeah.
[00:19:46] Andrew Zigler: the charge. So how, how at GitHub do y'all celebrate and create this environment for continuous learning?
[00:19:53] Martin Woodward: Yeah. we call it growth mindset here at GitHub. You know, it's a kind of a, um, a term I. think a few people to use, but it's sort [00:20:00] of, we want to be, a learn it all company rather than a know it all company. You know, we want to be consistently learning. Now, luckily, because kind of at the heart of GitHub is the open source community.
[00:20:11] Martin Woodward: A lot of people we hire kind of come with that attitude anyway. They assume that somebody out there is smarter than them, and So they, they can kind of, you know, figure out, like, do some research and figure out what the best way of approaching these things is. that said, like getting co like we built co-pilots.
[00:20:29] Martin Woodward: I've been using copilot, what, since 2021 ish. I think I've been using it. I've been using it for a while now. You know, it's like the, the, the sixth person to try copilot in the company and, it was cool, but like going from six people using copilot to, you know, the entire engineering org using copilot didn't happen overnight.
[00:20:50] Martin Woodward: what we find with adopting LLM tools is they, you know, the Moore's kind of innovation curve where you have like early adopters, then you have the innovation gap, and then you have, [00:21:00] you know, early majority, late majority, you know, like all that sort of stuff. Unlike DevOps deployments, and unlike kind of rolling out TDD in a team or rolling out Agile in a team, which is very much a,
[00:21:11] Martin Woodward: team by team one and done kind of way of deploying with copilot and with, I think with all AI tools, it follows this Moore adoption curve, 'cause individuals have to take the time to start using it in their workflow and start learning how to prompt the ai, learning what, where copilot can help. and so you get to 25, kind of 20.
[00:21:35] Martin Woodward: The people that listen to the show, they're all early adopters. And so they're the people that are just gonna grab copilot, grab any technology, give it a try, see how it can help. Amazing. 'cause they're listening to this show. But then like, if you think about all the people that you work with, like they're busy and they maybe don't wanna be trained.
[00:21:53] Martin Woodward: They've got an environment that works, they've got work, they've got an infinite backlog of work that needs to be done. Like when am I gonna find the [00:22:00] time to figure out how to use this? You know? And so helping them find that time and helping, them share when they come across something like you've obviously had some amazing moments where, you know, copilot or over LMS have kind of like surprised and delighted you.
[00:22:18] Martin Woodward: Finding a venue by which people can share those is invaluable inside of a company as well. We have a, I, I'll say stuff, copilot did like kind of channel where people can share like excitement or like amazement of like, I dunno what he just did for me. This is crazy kind of thing. Because people are gonna believe those people a lot more than they're gonna believe in me saying, 'cause you know, saying co-pilot is amazing.
[00:22:41] Martin Woodward: Of course you think that, you know, you're a vp, you just delete emails all day. You don't really code. Like, let me, let me talk to somebody who really codes. Yeah. So we do some of that. So, uh, yeah, but, but it, so we, we had to work on getting internal adoption even for us, and took us a while to get to the [00:23:00] point where the you know, the vast majority of devs are using co-pilot inside the organization.
[00:23:04] Martin Woodward: So if we did. Like be prepared that as you are building those tools inside your company, you are gonna have to do real change management. Like help people come along on this journey with you. Because what we see is that people are using copilot are like the numbers we have, you know, like what, 50% more effective?
[00:23:22] Martin Woodward: They're 80% happier. Like there's all these ridiculously big numbers that we have from doing surveys and stuff,
[00:23:27] Andrew Zigler: Yeah.
[00:23:28] Martin Woodward: which is cool. But then the people who are not using that in your organization, they're gonna miss out on that. Like that's not fair on them either. And they're not missing out on it because they're bad or lazy.
[00:23:37] Martin Woodward: They're missing out on it 'cause they're busy and they need the help to get on board. You know? So, um, Yeah. So that's why my, my biggest thing is be prepared for kind of it being an adoption journey, not an overnight. Everyone's gonna grab it and do amazingly well tomorrow, you know?
[00:23:54] Andrew Zigler: Yeah. No, it's, it's, it's really insightful to hear how like GitHub is dogfooding and adopting its own [00:24:00] tool. And when y'all were doing that process, and obviously
[00:24:02] Martin Woodward: Yeah.
[00:24:03] Andrew Zigler: in that process
[00:24:04] Martin Woodward: Yeah.
[00:24:04] Andrew Zigler: we're all still figuring out where it's going.
[00:24:06] Martin Woodward: Yeah.
[00:24:06] Andrew Zigler: things that you look at to really understand the impact or like the, the productivity or the developer experience gains?
[00:24:13] Andrew Zigler: Are there like go-tos that you really measure as you, as you have adopted this across your org?
[00:24:17] Martin Woodward: Yeah. the primary measure that we take is around like code acceptance, like how much people are, uh, accepting the suggestions from copilot and using them. And this usage as well is another key metric. Like, you know, are you, uh, a one day 28, a 10 day 28, a 28 day 28 kind of user? You know, um, how many people like a histogram of where we're seeing that happen inside the business and then what areas people are using.
[00:24:44] Martin Woodward: another metrics we're looking at are things like, well, that's cool. That's usage, you know, but like what's, what's happening to our PR merge times? What's happening to our security vulnerability? Uh, response times, you know, are we [00:25:00] seeing correlative improvements in those alongside the percentage adoption of copilot that we're seeing?
[00:25:07] Martin Woodward: And turns out we actually are like, even if I look at across all of GitHub, like every user on GitHub. I can see, but we, we introduced, you were talking about getting copilot into the hands of people. That's one of the reasons why we made a free skew of copilot available so that people can just try copilot and not have to ask permission.
[00:25:26] Martin Woodward: If you can just give it a go. when we introduced that, I, I can see the graphs, I can see like this marked, increase in the people using copilot. And this marked decrease in globally, not just in GitHub, but globally, the average PR merge time and also, an increase with the amount of PRs being created.
[00:25:46] Martin Woodward: Like it's, and it's, it is day in day, like it lines up exactly with that graph of copilot adoption.
[00:25:52] Andrew Zigler: cool.
[00:25:53] Martin Woodward: So the PR merge, so it's not just the, the number of prs going up is like, is cool. [00:26:00] but the size of PRs is going down and the amount of time it takes to merge a PR is going down. The amount of time it takes to merge, to fix a security vulnerability from it being detected is like gone from sort of weeks to hours.
[00:26:15] Martin Woodward: You know, it's like, it's, it's ridiculous how much it speeds people up that way. but in. again, back to our like old school things, all in positive ways that I wasn't sure at the beginning of this journey, like, would, would pr sizes go down? I was a little bit afraid they would go up because now LLMs helping me write, code me, I'm gonna write Monster prs. Yeah, exactly. But turns out no people need to iterate more and so their prs actually get smaller. So cool that seems like a positive thing, but not thing that, not something that we could have. Known for certain until we started on the journey, you know?
[00:26:51] Andrew Zigler: Yeah, it's almost like the global adoption of the AI that you can see on trends. It helped everybody move into a better development. Development work style, because we all know that [00:27:00] having smaller, very focused prs, you know, it's a best practice and being able to iterate on them, you don't want a massive PR that changes like eight different things.
[00:27:07] Andrew Zigler: So you see these like best practices get, like globally implemented as part of like the LLM adoption. So it makes me excited about like how much more efficient and safer engineering can get in the
[00:27:18] Martin Woodward: Yeah,
[00:27:19] Andrew Zigler: to like adopt these tools. which kind of
[00:27:21] Martin Woodward: and I think that's 'cause of the improvement in communication because, because you're, again, now to build, you're having to communicate more and so because you are, but 'cause you are communicating with an agent, with a, you know, with co-pilot rather than with another developer. that's what's forcing you to actually make those smaller prs and things.
[00:27:40] Martin Woodward: 'cause if, if copilot gives me a monster like change cause it's my, it's my job on the line. If that change breaks, like I'm the one that's gonna get called out. So a co-pilot's not gonna respond well, you know, sometimes it does, but it's not gonna, it is not gonna answer the bleeper at 4:00 AM you know what I mean?
[00:27:56] Martin Woodward: Like, I'm responsible for this code, so I have to do that code [00:28:00] review. I have to make sure it's good. And the only way I can do that is if I do smaller prs and make sure that this code is good.
[00:28:05] Andrew Zigler: it, it just goes to say that having those fundamentally good best practices is gonna help you move faster with these new tools.
[00:28:10] Martin Woodward: I think so.
[00:28:11] Andrew Zigler: I. And, and it kind of brings us to also to like a larger question of, of, or rather opportunities that I see within,
[00:28:17] Martin Woodward: Right.
[00:28:18] Andrew Zigler: building new technology, having this like democratization of development with
[00:28:22] Martin Woodward: Yeah.
[00:28:23] Andrew Zigler: seeing a boon around things like open source.
[00:28:25] Andrew Zigler: You know, Martin, you and I both like are from an open source world. We're both very, very, uh, very pro open source. And,
[00:28:31] Andrew Zigler: I'm curious to know your perspective on how you see open source contributions and communities evolving in an age of AI enhanced development.
[00:28:40] Martin Woodward: Yeah. one of the things that we're, I mean, we make co-pilot available, uh, so co-pilot is available for free to anybody. Uh, but there's limits in terms of like how many requests you make and things like that. So, you know, but for a hobbyist stuff, for doing open source work, it's plenty. But then what we also did is we made co-pilot, available co-pilot pro.
[00:28:59] Martin Woodward: Which [00:29:00] is our paid sku. We made that available for free to. It's like over a million kind of students, teachers, and, maintainers of popular open source libraries because again, we want to just get it in the hands of people, to help them improve how they can build open source. So, We, we do that. Uh, and what we're actually seeing is, you know, people really starting to use it.
[00:29:24] Martin Woodward: Like we've started enabling a few projects with the coding agent. and we're sort of scaling that out, and people actually, you know, using that learn, learning how to use it, learning what instructions they need and what actually the improvements they need to make to their builds and tests to then allow co-pilot to automatically run the builds and tests, which allow co-pilot to make better suggestions and things.
[00:29:46] Martin Woodward: Again, better DevOps. so we see a bunch of that happening. we see the ability to respond to pull, to the ability to respond to security defects actually dramatically improving from it as well because again, from an open source, [00:30:00] uh, maintainers maintainers point of view, they're busy. They're trying to like build stuff on the weekends and maintain stuff on the weekends.
[00:30:05] Martin Woodward: So if there's help available for me to do that, then that's great. And then the other area where it's actually hugely beneficial is code review. So one thing that copilot can do ridiculously well is tell me what this code actually does. Not what the human is telling me it did, it is supposed to do. and then spot the delta.
[00:30:29] Martin Woodward: usually from that, that helps you with a few ways. One is it sees what the code does and helps find defects, help find bugs where it can see what the human saying they intended. And actually what the code do does is a delta. This is probably this bit of code that's wrong, like somewhere. Um, but what it can also do is spot those.
[00:30:48] Martin Woodward: Um, uh, you know, there's, the human says this fixes an important security vulnerability in your library Mr. Open Source maintainer, copilot says this adds a Bitcoin mining, [00:31:00] uh, wallet stealer or something like that, you know what I mean? And then the maintainer is like, oh, maybe I need to take a look at this code a bit more than I was going to.
[00:31:07] Martin Woodward: And it makes it harder to kind of sneak those things through. Um, so there's a whole bunch of ways there. I think it can kind of help improve open source, but when we first introduced these tools, we did see a, it is calmed down a lot now, but we, early days we did see a spate of actually, open source maintainers getting quite a lot of, we, we politely call them low quality submissions.
[00:31:33] Martin Woodward: Some people will call them spam or whatever, but they're not really spam. They're usually by well-intentioned people pointing. an LLM at a project and saying, do something for me without the skill to know if the answer was correct or not. And then submitting it anyway, thinking they were being helpful kind of thing.
[00:31:52] Martin Woodward: And then because, you know, copilot does a great job and all LLMs do a great job of like sounding [00:32:00] authoritative, like the person, sort of the maintainers, gotta review that and take a look and like, what is this doing? It's harder to, it's harder to detect that the thing that this person's coming from isn't as skilled as it. might sound like they are based on their PR descriptions and things like this.
[00:32:15] Martin Woodward: So we had, we had to do some work there, a bunch of education. Thankfully things have got a a lot better, but it's something that we're definitely. you know, we're being stewards of the open source community, so that we're very, very mindful of is that we don't, we don't accidentally overload these maintainers with, you know, we don't make it too easy to overload these maintainers with low quality submissions kind of thing.
[00:32:39] Martin Woodward: So, you know, there's a bunch of work happening there to try and help improve that and like make sure that the tools are helping reduce workload from the open source maintainer and don't add too much workload to them.
[00:32:50] Andrew Zigler: Yeah, and one of the last things I wanna touch on here is
[00:32:53] Martin Woodward: Yeah.
[00:32:53] Andrew Zigler: that persona of that person who might pick up this tool and not have the coding knowledge. Your understanding of the code [00:33:00] base and, and, you know, use the LLM and the LLM
[00:33:02] Martin Woodward: yeah,
[00:33:02] Andrew Zigler: very, uh, authoritative. It's convincing and it's
[00:33:05] Martin Woodward: yeah.
[00:33:06] Andrew Zigler: into this like false trap of understanding what it is that you
[00:33:10] Martin Woodward: Yeah.
[00:33:10] Andrew Zigler: prompted through.
[00:33:11] Andrew Zigler: And that, like you said, creates friction. And some like things like open source projects. It also creates, opportunities like for junior developers of actually really understanding what they need to understand in order to have an impact. Right, and so in our, our listenership, we have a lot of active developers, a lot of
[00:33:28] Martin Woodward: Yeah.
[00:33:28] Andrew Zigler: there's also a lot of like aspirational developers and
[00:33:31] Martin Woodward: Yeah.
[00:33:31] Andrew Zigler: to try and use these tools.
[00:33:33] Andrew Zigler: And I'm curious, what would you, uh, in your position say to someone who's maybe more of a junior developer about where they fit into the developer world now and what they should focus on for being the best developer they can be tomorrow.
[00:33:45] Martin Woodward: Yeah. that's great. first of all, a warning to those senior devs. Like one of the, I see, I see a lot of snobbishness that goes around that's like, Oh yeah. I can use these LLMs, but that's not for the juniors, that's not for the kids kind of thing, or whatever. You know? That's not for the less experienced [00:34:00] people, because they're never gonna understand what this stuff is.
[00:34:03] Martin Woodward: They're not gonna know how to use it properly. I'm like, no. No, sorry. The, the people who are coming on are just as smart as you. They just don't have the experience that you've got. And, if you think about your, like I've been around, I've got no hair anymore for people that are not watching, like I've been around a long time.
[00:34:21] Martin Woodward: Uh, and when I, when I first came into, the difference between when I was building stuff at home as a hobby to when I came into the workplace is that I went from building code that I completely knew inside, out, knew everything, and I knew every single line of that code to being in a code base that was like, just didn't, like, there was bugs in this code base.
[00:34:45] Martin Woodward: It was written by other people. It was just did stuff and, and like having to understand what this code did. This next round of junior developers are gonna be a lot more experienced at that. And to be honest, that's how most of our real work is because the LLM helps [00:35:00] create the scaffold for 'em, helps create some work, and then they're trying to figure out where it doesn't work.
[00:35:04] Martin Woodward: Like they're trying to figure out the debugging skills, they're trying to figure out all those sorts of things. So actually, in terms of the work that we do, I think they're in an ideal place in terms of you answering your question around like what skills. learning about the different technology stacks, different architectures, learning about what different languages are capable of doing so that you can.
[00:35:26] Martin Woodward: There's less of a barrier now to go from say, Python to Java, to go from like this ama, you know, whatever the JavaScript framework is that you are familiar with to a more recent one, to A to one that's more appropriate to this particular task that you are using. So the more you can build up your knowledge of what frameworks are out there, what languages are out there, and what are the right, types of solutions for these types of tasks.
[00:35:52] Martin Woodward: I think you're then better positioned to then be able to guide the AI to give you the right kind of solution rather than doing what we did as [00:36:00] developers, which was, I know how to build one of these. How do I solve a problem with what I know how to use? You know what I mean? So it's a great time to be a junior Dev, I think, obviously.
[00:36:09] Martin Woodward: Um. You know, bearing in mind like all economy shifts and everything else like that But, um, in terms of the skills, like if I was a kid now learning to code or if I was, you know, switching careers and learning to code, the fact that I can like highlight some code and have it explain code to me in my language is amazing.
[00:36:27] Martin Woodward: Like, I had to go to, I had to order books from the library and wait two weeks of them to arrive and then read the books. you know, like now you just highlight code. Explain this.
[00:36:36] Andrew Zigler: to the problem, right? You had the guess of like how that
[00:36:38] Martin Woodward: Yeah, yeah, yeah. Exactly.
[00:36:40] Andrew Zigler: Overflow where you did the same thing. Oh, how does this person's question actually relate with what I was doing? And so being
[00:36:46] Martin Woodward: Yeah,
[00:36:46] Andrew Zigler: personal answer is, is
[00:36:48] Martin Woodward: but, but also learning that it's an unreliable narrator. That it's as fallible a narrator explaining that code to you as a human is. And that then helps you as you're building your career, because I'm sorry, [00:37:00] I, I probably give just as many wrong answers as I give right answers when people ask me questions, you know? And so, it helps you build up your internal monologue and your internal skepticism of like, okay, how can I prove this? How can I test this? How can I break this down into small chunks that I can build up that are testable, that I. can increment with kind of thing.
[00:37:21] Andrew Zigler: Yeah, I think that's a great way of looking at it. And, and
[00:37:24] Martin Woodward: Yeah.
[00:37:24] Andrew Zigler: I, I, I definitely take that to heart. I completely agree. With you and Martin, we're coming up on our time here and we've had such an incredible chat about where the future of development is going, how y'all have been building, GitHub agents and co-pilot and, and, and,and your own opinions on, on how DevOps practices are able to kind of accelerate that adoption.
[00:37:43] Andrew Zigler: I can completely agree. Agree with them. I know many of our listeners will as well. as we wrap up, where can folks go to learn more about stuff that's top of mind for you? Obviously, everyone knows where to find GitHub, but are there
[00:37:53] Martin Woodward: Yeah.
[00:37:53] Andrew Zigler: that you wanna point them towards that you think are really salient?
[00:37:56] Martin Woodward: Yeah. Well if you wanna, um, GH slash [00:38:00] copilot and we'll provide links in the show notes is, is a link to kind of all the copilot stuff so you can learn what's new and then follow along with the GitHub social so you can kind of, the, the pace of change is ridiculous. Like we add new models every week it feels like.
[00:38:14] Martin Woodward: And so keeping up to date with what shipping is probably key. Folks can also, feel free to, we've got a conference coming up called Universe, uh, towards, you know, in um, October timeframe, there'll be a bunch of cool stuff shipping. Then there'll be lots of announcements then as well. So I would, I would, uh, encourage people to look out that.
[00:38:32] Martin Woodward: And then finally, like Martin Social is my Blue Sky handle, but it's also my website that links to all of my socials and things so people can feel free to just go there and, um, and figure out where to follow me if they like. a British kind of accented to their tech news and that's always good as well.
[00:38:49] Andrew Zigler: Great. That's awesome. We'll, we'll make sure we get all this in the, in the roundup so folks can go check it out. And GitHub Universe? Is that what you said is, is later
[00:38:56] Martin Woodward: Yeah,
[00:38:56] Andrew Zigler: right? Is the big one? Yeah, I'm, I'm pretty keen for that one. [00:39:00] I'm, I'm excited to see what happens. May, maybe Dev Interrupted will be at that one too, so I'm
[00:39:04] Martin Woodward: be great to see. you in person. That'll be amazing.
[00:39:07] Andrew Zigler: maybe we could, maybe we could do this in person and, and, and
[00:39:09] Martin Woodward: That would be lovely. '
[00:39:10] Andrew Zigler: cause, like what you just said,
[00:39:11] Martin Woodward: I.
[00:39:11] Andrew Zigler: going so fast. So,
[00:39:13] Martin Woodward: man.
[00:39:14] Andrew Zigler: we we're gonna have to keep talking about it, I think. 'cause the, there's just so much more to uncover. So folks, thanks for joining us today. Uh, this has been a really fun chat.
[00:39:21] Andrew Zigler: If, if you enjoyed it, made it all the way this far you clearly need to like the episode, you need to share it and you need to subscribe. But more importantly, you need to go on socials and you need to tell people what you thought about today's conversation. You should tag Martin and I we're both active on places like LinkedIn.
[00:39:36] Andrew Zigler: Uh, it's really easy to find us contribute to the conversation. We wanna know how you're using these tools and, and what are the conversations we need to have to make it more effective. so thanks for joining us and as always, stay building and we'll see you next time.
