What do the teams at Stack Overflow, DataStax and Reprise have in common?
First, they’ve all built amazing organizations powered by amazing developers.
Second, they’ve managed to build and retain these amazing developers in an ultra-competitive hiring market.
Third, they all took time at our recent INTERACT conference to discuss how they created engineering organizations that are productive, successful and - best of all - happy.
It’s a rare treat to have this many amazing minds on one live panel and we couldn’t be more excited to share their insights, wisdom and advice to the community at large - so get ready to hear from Ben Matthews, Director of Engineering at Stack Overflow, Shankar Ramaswamy, Head of Engineering at DataStax, and Jennie MacDougall, Director of Engineering at Reprise.
Quick note, if you notice a tiny fluctuation in the audio quality for each speaker, it’s because this was recorded live at INTERACT.
Enjoy!
Episode Highlights Include:
- (7:29) Three stages of communication
- (13:39) No dev works better when they are stressed or burned out
- (17:19) "Platformization" what it is and when to do it
- (24:05) Allowing engineers to test run a new job
- (37:09) Velocity is a data point, not the story
- (43:06) Building teams around people and product
Our Interact conference featured content from the best engineering leaders in the world. Watch it on demand on YouTube now.
Transcription:
Conor Bronsdon: Host
Ben Matthews: Director of Engineering at Stack Overflow
Jennie McDougall: Director of Engineering at Reprise
Shankar Ramaswamy: Head of Engineering at DataStax
[Music Playing]
Conor: [00:00-01:00] Hi, this is Conor Bronsdon, executive producer of Dev Interrupted and your host for this episode. What did the teams at Stack Overflow, DataStax, and Reprise have in common? First, they've all built amazing organizations powered by fantastic devs. Second, they've managed to build while retaining their stellar dev teams in an ultra-competitive hiring market. Third. They all took the time at our recent interact conference to discuss how they created engineering organizations that are productive, successful, and best of all, happy. And we couldn't be more excited to share their insights, wisdom, and advice to the community at large. A quick note, if you notice a tiny fluctuation in the audio quality for each speaker, it's because this was recorded, live at Interact. Enjoy!
This episode is sponsored by LinearB accelerate your development pipeline with data-driven engineering metrics, continuous improvement automation, and project visibility while cutting your software development cycle time in half. Sign up for your free demo at linearb.io and mentioned the Dev Interrupted podcast discount for one month free when you sign up for an annual pro membership.
Conor: [01:01-01:24] Hey everyone! Welcome to our panel on how to build a successful engineering organization. I'm Conor Bronsdon, one of your Dev Interrupted community leaders and I'm delighted to be here with three incredible engineering leaders. We've got Jennie MacDougall, director of engineering at Reprise, Shankar Ramaswamy, head of engineering at DataStax, and Ben Matthews, director of engineering at Stack Overflow. Welcome the panel, everyone.
Ben: [01:25] Thank you.
Jennie: [01:26-01:27] Thanks so much. Excited to be here.
Conor: [01:28-02:04] Yeah, I'm really looking forward to this conversation, I think there's going to be some great back and forth and I know we're going to talk more in depth on communication of processes, technology and how to keep your engineers productive, plus topics like culture and building trust but I want to start this conversation by kind of posing a hypothetical question to help warm everyone up and get to know one another. Let's say that all of us have to start an engineering organization from scratch with the leaders here on this panel. And the three of you only get to bring one thing to the table, one process or piece of culture. What is that piece that you're bringing to the table? Maybe, Jennie, if you want to start.
Jennie: [02:04-03:01] Sure. I would say that one process we have is we do engineer roundtables weekly, and that is essentially a meeting that is run by one of our engineers, really any subject matter expert, and it's really a technical deep dive into a new piece of code, a new epic, a new feature. Really anything that they want to share with the rest of the team. We share why we chose it, how it's implemented and how it's intended to be used. We identify any new patterns, any changes to existing patterns that folks should be aware of and really any gotchas that engineers might run into when they start using it. It leads to great discussions and allows engineers to ask questions and really kind of dig into it. Also, helps create a more consistent code base, especially being remote, helps to create a more consistent code base and allows engineers to really better understand the architecture and kind of the vision of the company from different points of view.
Conor: [03:02-03:06] I really like that. Ben, what about you? What's what are you bringing to the table here?
Ben: [03:07-03:43] You know, not as a cop out, but for a lot of these things, it all starts with like “it depends” right? But if there was going to be like my blanket statement of what I want to bring back a lot of my successful organizations, one thing they have in common is and maybe kind of in a sense of what Jenny was mentioning, like ways to facilitate collaboration, open communication channels, and a positive feedback structure. Like-like receiving feedback is just as much of a skill as giving feedback and just places where we've been able to have that open line of communication and collaboration and it really just kind of organically grows into everyone just getting better at what they do.
Conor: [03:43-03:46] Shankar what's your contribution here?
Shankar: [03:46-04:45] Yeah, so I bring sort of my-my three guiding principles to the table in terms of how the team works and makes decisions. I call them accountability, autonomy and alignment. So, the idea is that everybody's clear on what problems they're accountable for and they own. And then everybody has-is empowered to make the decisions that they think are the best decisions to solve those problems and deliver on that accountability, that's the autonomy. And then the alignment is folks understand and have enough shared context. So, when they're making independent decisions, they're coherent and consistent with other folks who might be making similar independent decisions. Right? And so that, I think for me, sets up teams, you know, with the right structure so that you can scale and go fast and-and yet continue to work together effectively.
Conor: [04:46-05:04] I kind of hear that same thread of a lot of what all three of you said right? Jenny, the process you're talking about these deep dives that are held on a weekly basis, that's definitely an opportunity for alignment and to make sure the team gets on the same page and shares knowledge. So, I hear that thread through all three of you.
Shankar: [05:04-05:13] Absolutely. In fact, we have we have a weekly deep dive is one of the mechanisms we use for alignment. So absolutely, plus one to what you just said.
Conor: [05:14-05:22] And if I recall correctly, Shankar, you also do AMAs with your team to make sure that they have transparency and to kind of build trust throughout the team.
Shankar: [05:23-05:53] Absolutely. Absolutely. There's-there's a whole bunch of mechanisms we have to-to kind of create this alignment and trust and everything and the shared context, right? So, a lot of those AMAs are designed to both-to help me hear from folks, right, in terms of what problems they're facing or what challenges or what they're excited about, right, all of those things. And then more importantly, for me to share context with them that I might have, you know, that-so, it's both a gathering context from everyone as well as sharing context, kind of.
Ben: [05:53-06:10] I love the fact around the gathering context as well. I think it sometimes is kind of overshadowed by this focus on Top-Down communication. But even in those AMAs even the questions asked, you're learning so much around where their focus is and where their minds are so you can react and assist there.
Shankar: [06:11-06:12] Yeah, absolutely.
Conor: [06:13-06:24] So, I hear a lot of threads around communication here. It's very clear that's important to all three of you. And I think we all agree that's integral to a successful engineering organization. How do you communicate with your team members when something goes wrong?
Jennie: [06:26-07:19] I think one of the most important things, when things aren't going perfect, when you do have to have one of those harder conversations, making sure that the other person is ready to have that conversation, is ready to listen, is not distracted, that can be as simple as a “How are you?” or is still-“Is now still a good time?” and just how they respond to that quick question will let you know, okay, we can have this harder conversation and not maybe start having it and they're distracted by something or they're not able to listen at that moment. Everyone has kind of their lives happening in their background. So, making sure that it is a good time and they are able to kind of take the time to understand what you're about to say, I think is probably one of the most important parts of it.
Conor: [07:21-07:28] Definitely agree with that. And I hear kind of that empathy thread coming through here around like making sure you have team member empathy and build that trust.
Shankar: [07:29-08:48] Yeah, I think-yeah I think a couple of other things that add to what Jenny said is probably there's like three-three different stages of talking, right? One is talking about the fact that things will go wrong, it's just a matter of when right? So that's something that I like to emphasize all the time so that people are not worried if things go wrong that, you know, oh my God, this is unusual or unexpected. Right? So just-just, you know, creating that safety for-for folks to think about it. And then when things do go wrong, giving the team the space to work through things, right, and not sort of imposing myself or, you know, other people coming in asking questions or creating, you know, more stress and pressure in the system, right. So just kind of giving folks the space to do it. And then, as Jenny mentioned, after you know, we've kind of gone through the issue and resolved it, at the appropriate time, going through that process of introspection and reflection to say, hey, why did it go wrong? What can we do better? And that type of thing. And again, letting the team lead, right? Rather than, you know, doing a lot of you know, questioning or drilling people. Just kind of let them talk, let them reflect, let them realize what's-what's gone wrong.
Ben: [08:49-09:53] Yeah, I think you both touched on kind of two different aspects of-of when things go, I think. And Jenny, you more on like the individual level of where I think-actually maybe to go back to a little bit of what you said before Shankar, on just like setting expectations. I think like once those are clear and people understand what you're hoping for and what they need from you to be able to achieve those-those conversations go a whole lot easier. But also, as you were saying, Shankar, like when it's like maybe a platform issue or an application issue. You know, one thing that we were stressing around that was this-and it's kind of pervasive throughout, but even around that, is the psychological safety of something's going to go wrong and-and it's okay to react to it. And kind of the saying I have with my team is like the psychological safety for the team is “You feel safe to ask for yes, but you're not crushed if the answer's no.” And I think having a lot of those questions come up around like systems stability or am I doing this right? Is this a crazy idea? Kind of even gets ahead of those and having those questions already out there when something does go wrong helps us get-get ahead of that curve.
Shankar: [09:54] Yeah.
Conor: [09:54-10:05] So, what are other team commitments that you would want to see put in place? Shankar, I know you mentioned you have your three A's: account-accountability, alignment, autonomy. Are there other commitments that the three of you are making for your team?
Shankar: [10:05-11:17] Yeah I think we have also at DataStax, a set of values that that we're all, you know, looking to practice and live by. We have four values. One is around, you know, obsessing about are developers and enterprises who, you know, we are here to serve. Diversity and inclusion, right? Making-making sure that we're creating a diverse and inclusive workplace. Executing, you know, well and being an owner, right? Those are sort of our four values. And so, what we've done is we've taken those values and also talked about specifically how we would apply those in engineering, right? So, for example, taking something like obsessing about developers and enterprises, we talk about, you know, thinking about outcomes we are trying to drive. We are a database company, developers are using us, you know, to-to build applications. So we want to make sure we understand what outcomes developers want. And we're, you know, working towards those outcomes. We're also looking at details because details matter, respecting the details. And then finally, we're nailing the experience. So that's kind of concretely how we are asking engineers to think about that value, living that value. Right? So that's-that's something else we do as well.
Conor: [11:19-11:23] And Ben, I've heard you talked about how you build a culture of trust within your engineering team.
Ben: [11:25-12:36] Yeah. I think it kind of comes back to especially the humility point that we talked to before. I can't even understate how-how big of a reaction we get when we see very senior folks or even myself say, “Yeah, I made that mistake. That is my fault. This is how I'm going to work on trying not to do it again.” And especially throwing in that last part of like and this is what I'm going to try to do to make up for it, to make it kind of actionable. It kind of filters down into even other things of that it's okay to, to make these mistakes, try something new, but also even filters into how people can take care of themselves. One of the biggest shocks one of my new hires had was our commitment-like “You're on vacation. Please stop answering your email. Why are you on Slack?” And I don't go as far as like to turn off their access or anything, but I'm like, this is your time. And to try to set an example of that. When I'm gone, I’m gone. I set a message, “I'll respond to you when I get back.” This is kind of my time and it's safe for them to do that. They don't have to feel like someone's wondering like, “Where did they go? Why are they not doing their job?” And just kind of setting that example of like, I'm taking care of me and I make mistakes. I'm human. I expect no different from you. That kind of gets pervasive throughout the whole team as they see their peers doing the same and their leaders.
Jennie: [12:36-13:36] Absolutely. And even at Reprise we-it definitely is kind of set-the example is set by the execs and flows all the way down. Our motto is “Work hard and go home.” That can be difficult when your-when your office is the same room as your kitchen and your bedroom. So really encouraging folks to unplug. What that exactly means for them obviously varies, but not Slacking folks or expecting a response after hours, on the weekends. When you're on vacation, don't look at your email, don't answer slack. Our CEO went on vacation for a week and a half, and we didn't bug him and he didn't check in on us. And that was fantastic to see. And it showed everybody in the company that, like, we really value your personal time and really unplugging to recharge so when you come back, you're ready to go.
Shankar: [13:37-14:24] Actually, I just wanted to add one thing we're doing at DataStax this year is we've actually declared a number of refresh days for the entire company. So, one of the things that, you know, kind of is nice about taking synchronized time off is that you don't have a catch up right afterwards. And so. we've got every quarter; we've got like an extra-long weekend, right? Like a four-day weekend. And then two times in the year, once in the summer and once around the holiday season, we have like an entire week that the whole company’s not working, right. So that, you know, we tried a limited experiment last year and that worked very well. So, we're doing that a little bit more. And that way, you know, everybody is kind of off. At the same time. There's no question of anybody sending emails and things like that so.
Ben: [14:24-15:16] I'm not sure what’s your criteria, but when you said it works very well, I think like a lot of successful organizations are seeing benefits from that when I think sometimes it's still framed as like-as a sacrifice or something generous we're doing for them. But it's a-it's a win-win. I've never met any of my devs that have worked better because their stressed or burned out. This is-this is helping us. It's helping you. It's not something that we're giving you it’s we're investing in you as you can perform for us. So, I think we're going to start seeing that more and more. As you know, the-it's a very difficult climate outside of work right now, as well as inside work. So, I think as we keep finding these opportunities to let people take care of themselves and increase in performance and just the culture that comes from that, I think we're going to see a lot more organizations doing that more and more.
Jennie: [15:17-15:44] Absolutely. And I think there's different kind of flavors of it. Shankar, you mentioned a few four-day weekends and a week throughout the year. We actually do every month, everyone gets a three day weekend. So, most holidays fall on Fridays or Mondays so those are kind of built in. But if there is a month that doesn't have one of those, we create one and just make it a Reprise holiday. So, there's just something great about a three-day weekend.
Shankar: [15:45-15:48] Yeah, absolutely. Absolutely, yeah.
Conor: [15:49-16:06] I like that all of you think about building a sustainable engineering model. And Ben, I know I've heard you say this phrase before, like effective workers are well rested ones. There's a very reciprocal exchange here. It's not just something, as you mentioned, that we’re, you know, giving to workers to be generous. There is an effect that's positive for the company.
Ben: [16:07-17:04] Yeah. And they think it keeps paying dividends. And a lot of the things that we try to track that, you know, hard to kind of but that we don't have a direct control over. But like the-the length of time that people happen to stay on these teams that I have that we really invest in their psychological safety and well-being. They stay for longer. And I think we can all agree right now the-the dev hiring situation out there is very competitive, very volatile. But when you kind of look at it like just money isn't even the number one thing for most. For some, it's not even in their top three. They're worried about a place where they can enjoy what they do, where they can learn where they can-can grow. And they're not worried about 3 a.m. calls because something broke These are the things that I think can really set companies apart instead of just throwing dollars at it. And the more I just kind of invest in these and the more with more teams I have the longer they stay, the happier they are, the more and the faster they seem to grow. It's really been beneficial.
Conor: [17:05-17:09] What are other ways the three of you are looking to build this sustainable engineering culture?
Shankar: [17:10-18:50] Yeah. So actually Conor, that-that phrase, I've actually thought about it a lot. And one of the models that I've come up with and we are implementing at DataStax is even looking at how we spend our bandwidth, right. See, typically what happens is we can get overly focused on sort of new products and features, right? And not think about other areas like engineering excellence, tech debt, right, or even product excellence. Like we put features out, but we don't take the time to sort of go off and finish them and get them, you know, 100% there, right? Or even platform-ization, right? Often, you know, you'll find that you're doing the same thing over and over where you'd be better off with a framework or a platform that would give you that long term reduced cost of maintenance plus, you know, velocity with new things. Right? So-so, one of the things that we've been doing is looking at consciously balancing our investments across all these buckets so that we are actually not just focused on, you know, new things, but we're also taking care of, you know, existing things and taking care of tech debt and all of that. And that then makes for this kind of long term sustainable, you know, effort, right? Otherwise, if you go too far down just doing features and products, then there comes a point where you're spending pretty much all your time just keeping the lights on, right? Because you've got so much debt, you've got so many things that-that need attention and so you're just constantly on the treadmill, just keeping the lights on.
Ben: [18:51-18:54]And then even the speed to build those new features slows down more and more.
Shankar: [18:55] Exactly, exactly.
Ben: [18:56-19:26] Yeah. I love what you're saying about the platform-ization of it, if you don't mind me asking, like, you know, we're doing that now. We have a big tech investment year actually ahead of us. We got great agreement from product that we're going to spend a significant amount of time and working on the platform. But there is kind of like-what would be a rubric for when, okay, this is a good thing for us to platform-itize, to turn that into a verb somehow, but-because you can go too far the other way, sometimes of you're trying to make everything ubiquitous and then nothing works the way you want it to.
Shankar: [19:26-20:10] Yeah, no, absolutely. Thank you. And by the way, this is you've-you've-you’ve struck a-struck a chord with me because I've spent the last twenty-five years building platforms of all kinds. So, I think-I think your question is spot on because there is what I would call premature platform-ization, right? Which is, you know, people starting too early to build a platform. Right? Because then you start dreaming up all these different scenarios and use cases. And then you can go nuts, right, and waste a lot of time. I think platform-ization is a good thing to do when you see you've got a repeated thing happening multiple times, right? So, you've got three, four and you know there's more coming, right?
Ben: [20:10-20:13] When you say happening, you mean like multiple teams doing the same thing?
Shankar: [20:14-21:09] Right, right, exactly, exactly. So, you see something sort of a pattern repeating itself across different teams, right. So, for example, you know, a good simple one to understand is let's say you're launching a service in many countries, it's very likely that there's going to be a lot of commonalities there across all of those countries, right? So, then you might say, hey, you know what? Let's kind of build a platform where we have some of those things kind of put together in one place, and then everybody can sort of add whatever flavor they need to in each country. And so, I think you want to see enough examples to allow you to conceive of the platform correctly. Right. And then you also want to see a future roadmap that includes more of these instances, like the platform being leveraged. Right. So-so you want to see both of those things. You want to know enough to build the platform, right? And then you want to know that that platform is going to get used a fair bit in the future. Right. And then you want to invest in a platform.
Ben: [21:10-21:31] Yeah. I think a lot of these platforms that we've seen that-that we've tried to build and some of them very successful, but some like we try to over optimize or make it too flexible, so that then like all of the logic heavy lifting is moving to the client-the consumer of that platform anyway. So yeah, it's always kind of a fun to see where that balance is from organization to organization.
Shankar: [21:31-21:32] Absolutely. Absolutely.
Conor: [21:33-21:38] How do you measure the impact of whether you’re platform-itize something or taking on new features?
Shankar: [21:38-22:08] I would say your-your long-term maintenance costs should go down. Because now you've got you know, end implementations that you've been maintaining and hopefully you have most of it in one platform plus, you know, end sort of customizations, if you will. Right? And then the second thing is velocity for new ones, right? So, when you're doing the and plus one and plus two, that should start to be much, much quicker for you. Right? If you've done the platform right. So-so, I would say those two would be key things to look at.
Conor: [22:10-22:34] I'm curious. I'm hearing a lot of threats around how to sustain the team, how to set the team up for success in the future. What about setting up individual career success for developers? Obviously, that's a huge part of maintaining devs in an organization, enabling them to want to continue to succeed and grow. How are you helping team members measure their individual impacts and giving them opportunities for growth within your org?
Jennie: [22:35-24:03] So, I think there's two ways to look at it, both quantitatively and qualitatively. Kind of depending on the seniority or the role of a certain engineer. It could be individual velocity is really kind of what they're going for and making a really big impact that way. At the same time, individual velocity might decrease because they are more focused on code reviews and mentoring the more junior folks on the team. So, I think it really depends on what the career goals are. It also, I think, is important to figure out kind of-or at least ask them what they want to be when they grow up and allow them the space to kind of figure that out and create opportunities for them to figure that out in in kind of a safe environment. One example is if an engineer is thinking, maybe I want to manage some other engineers one day, let them manage someone on their team, or at least give them an opportunity on that team to kind of dabble in that and see if it's something they're actually interested in. It could take two weeks for them to come back and be like, “Yeah, that's a bad idea. That was terrible” So really being able to let them explore some of their passions, some of their interests in kind of a safer environment might be-or is something that we like to do.
Ben: [24:04-24:21] Yeah, I’m very-if you don't mind me ask you more about, that's a kind of a bold thing that, you know-kind of novel of having it-like giving someone kind of a test run of actually managing someone. I would love to hear more about successes and failures in doing that or how you set that up. That's kind of creative. I've never thought of that before.
Jennie: [23:21-25:17] Yeah. So, at Reprieve, and at my previous companies, we've partnered really well with Northeastern University and their co-op program. So, we have some junior engineers coming on board. It's essentially six months where they are working full time for us and allowing kind of the more senior folks to not-the daily-the daily management of them, but more from the technical side just to see kind of what it's all about and how it goes. One example is we had one engineer that was interested in the managing and literally after one sprint after two weeks, they came back and they're like I wasn't able to actually code anything. I was in meetings. I was figuring out what the team was doing and all these things. And I'm like, that's, that's kind of what management is.
Ben: [25:18-25:19] Yeah, welcome to our world.
Jennie: [25:20-25:37] It’s a lot less hands-on-keyboard and a lot more conversation. So just letting them kind of dabble in that before maybe jumping to a new company where that is their new position. Kind of letting them figure out if that's something they're interested in and [crosstalk 25:36] test it.
Ben: [25:36-25:57] Yeah, that's a great idea with the interns because-because especially where the interns would-like for the ones I manage, most of their questions are around good answers that senior engineers would have anyway. Like it's not as much traditional people management questions as much as “How do I become a good developer questions?” And they're the perfect people to model that. That-that makes a lot of sense. Thank you.
Conor: [25:57-26:06] Ben, I know you're also trying to build up more kind of rungs for junior engineers to give new people access to the industry. How are you going about that at Stack Overflow?
Ben: [26:07-28:14] Yeah, we-you know, there's-kind of what Stack Overflow does, kind of are the core of what we do is we really want to help developers where there's a little bit of an area for us to improve is that we like a lot of the questions and things we talk about at Stack Overflow are usually not like 101 questions of like “How do I JavaScript?” like that doesn't really go well in Stack Overflow. So, trying to invest in how we can help other people get into an industry, we're actually exploring some opportunities now that I'm not allowed to share, but we are very focused on that from a business point of view. But from even an individual point of view, we have restarted our internship program We're working with a great organization called Tectonica that's trying to help people that may be underrepresented in the industry and letting them-kind of give them an avenue to get exposed to professional development team and trying to get the skills there. Because especially when working with an intern, it's not even-like it's important to get exposed to code and how to ship it, but it's really like, how do I work within a development team? What are the questions being asked? Like, how do I handle merge requests? How do I do-review code? Like, these are things you don't learn in school as much that are vital to operate within the team. But-and even then, it's just kind of when not even speaking to folks outside of Stack, it's just letting them know to get over that intimidation hump. Because if you're someone that has not coded before, looking at this wall of funny characters is very intimidating. Like you like this looks like a foreign language and I'm supposed to know how this works. Getting them over that intimidation, starting small and just telling them to stick with it for a while, I think is a great way for people to realize that, you know, this is something I feel like anyone can do on some level. It's really hard to be a terrific engineer, but to get into engineering, even as a hobbyist, it's a very welcoming place right now. Tools are free, frameworks are free. There's so much information to to-to get on YouTube from Udemy or Stack Overflow just to kind of, like, look at all these answers. I just really encourage people to give it a shot. There's very little to lose compared to me being a curmudgeon in my old days of like, I had to pay this much to get an ID and this much to go to a course and buy a book. And all of that is free. It's a wonderful time to get into development.
Conor: [28:15-28:29] I love that philosophy. What about folks who are maybe later in their careers, how can we enable them to grow and expand their skillset or kind of take that next step if they, let's say, don't want to get into management and want to be, like, “No, I want to be like a senior staff engineer.”
Shankar: [28:30-31:35] Yeah, so at a DataStax, one of the things we did is we didn't have a good framework for folks to understand, you know, and appreciate what growth looks like and what they would need to do to grow. So, I spent a bunch of time last year and we rolled this out to the team late-late last year and early this year where, you know, we talk about growth along sort of three-three aspects, right. So, one aspect is the scope of responsibility that you're able to take on. Right? So, if you think about it, you know, when we when we start our careers, we usually can take on, you know, smallish, you know, straightforward kinds of things. We need a little bit of help and so on. Right? And then we progress to being more independent. We can take on sort of complicated things and handle them independently. Then we go on to taking complex things. We can even direct a few people, right, and break things down. And then there comes an important transition point of getting into more open ended, ambiguous things. So that's a very critical transition point. And, you know, relatively small number of people kind of move past, you know, from complex to open-ended and ambiguous and then-and then you get to like doing ambitious things. Then you get to visionary things and transformational things, right? So that's when you sort of go along that progression. And so we sort of talked about what that looks like, right? And-and then also talked about what sort of things would we look for when you're solving problems, what approaches, right, you know, what's a methodical approach look like? What are the various aspects of taking a methodical approach to, you know, are you using data? Are you bringing people along? Are you creating, you know-are you looking at alternatives? Right? So, all of that kind of stuff we talked about and then we also talked about specific areas, like if you're a program manager or you're an engineer, you're a technical lead, you're a, you know, people-people manager, what are the skills that you would want to look to develop? Right. Both kind of technical and non-technical skills, because a lot of times the non-technical skills are glossed over. Right. And especially people skills, right? People skills are something, you know, we're all not trained on, right, If you will. We get a lot of training on problem solving skills. But, you know, very rarely do people actually go and study psychology or, you know, anything else about people, right? So-so I think we talked about all of that and that's actually helped the team understand, you know, where they are at present and why they're there and then what they would need to need to do to-to get to the next level. And what we are doing is now finding folks who want to stretch and much along the lines of what Jennie was describing we’re actually then teeing up those opportunities. We’re saying, “Hey, it looks like you want to stretch. So let's figure out how to make an opportunity for you.” And the way our promotion process works is we-it's a post facto thing. So, you show us you can do the next level job, and then we'll say, okay, now we make it official and we give you the next level role, right?
Ben: [31:35-32:32] Yeah, and maybe this is also kind of harkening back to what Jenny said, kind of for people that are on the senior level trying to get higher, you know, like velocity is an important data point, but there's so many ways to interpret it that one of the things as people go up, we don't focus so much on individual velocity as much as the team velocity that they're on. We really feel like people are becoming like next level when they become a force multiplier for everyone else on their team. And even another factor of is kind of just how quickly it can go from idea to something that is workable. It's something you can kind of see by certain people on the team of like doing the research and kind of getting ahead of potential pitfalls in an idea. These are things that kind of just come with time and experience and kind of being put through the wringer of doing this over and over that they're able to have that expertise empower other steps and other people on their team, That really sets, I think, like the senior staff principle level engineers apart for us.
Conor: [32:33] Yeah.
Jennie: [32:34-33:18] I agree. I think that the impact that somebody has on a team is one of the most important things. Being able to both kind of explain from a mentoring side and then really be able to multiply that out and add value to both just their team and the organization as a whole. And we actually use a career lattice. So, it's not the typical ladder of you have to go in this order kind of up the chain, but it's kind of choose your own adventure giving folks different opportunities to see what they like. And even sometimes more importantly, see what they don't like because figuring out kind of what your real passions are and being able to follow those is so important.
Conor: [33:18-33:49] I'm hearing a lot of great takeaways here, but one thing I really want to dig into is this like measurement of impact side. You know, we lead with the topics of people and culture first for a reason, because too often we over optimize on tools development at the expense of other important concerns and because we all believe that putting the human at the center of engineering is crucial. I know that's a shared value here. But technology and data do have value, too. So, Jenny, I love to lean in. How is your piece leveraging data to continuously improve your engineering team?
Jennie: [33:50-36:04] I would say two of the biggest data points that we use, our sprint velocity and individual velocity. I definitely agree that the individual velocity is not as important. I do think it provides a lot of value though. It allows you to have more detailed and in-depth one on ones. I think. If someone's velocity is not kind of up to par with some of the other members on the team, it's a great conversation starter. Obviously not everyone has the same skill set and that's what you're looking for in a team that everyone brings something differently. But being able to use that data and kind of drill into maybe looking at their work and why are they only taking-everyone on our team at least, everybody's as a full stack engineer, why are they only taking the back-end stories or vice versa? And one example is one of my engineers. I realized they were focusing mostly on the back end and drilling into that a little bit realized that they didn't want to take a front-end story. It would take longer for them to do because they weren't as familiar, and they were afraid of letting the team down if they didn't hit the sprint commit. So, through that discussion, I was able to kind of figure that out because they aren't always aware of what their challenges really are. It's through conversation to kind of figure out what they really mean when they're saying certain things. But I then pretty much challenged them that every sprint they have to take one front-end story, take it early in the sprint, so they have plenty of time to complete it. But through that, they got a lot more comfortable with front-end. They were able to pair with someone who was a lot more experienced and comfortable. Now they have a great working relationship and rapport, and they were able to collaborate and really kind of work off each other. And now he's actually psyched to work more on the front-end. So just using those data points of individual velocity to figure out kind of where they're struggling helps. And then sprint velocity is obviously important to be able to roughly estimate what can get done in a given sprint.
Conor: [36:05-36:23] You're using-you're not using velocity in two ways, which are crucial. One, you're not comparing across teams so that you don't have this kind of gaming of the metrics that you mentioned. And two, instead of using velocity as like a performance metric of like, oh, you're not going faster enough, you're using it as like an indicator of like individual health and team health that you can then kind of dive into.
Jennie: [36:25-36:53] Absolutely. And every team is-pretty much has their own velocity. We are not comparing across, as you mentioned, and it's definitely used more as a tool to help figure out how we can improve rather than a metric to scold someone for not going fast enough, which obviously, at least in my experience, has never worked out. So, using it to help them out rather than deter them.
Ben: [36:54-39:20] I just want to say I love that story of that person getting more into the fire, creating that psychological safety again of like, go for it. I got your back, try the front-end. And I don't expect you to be a rockstar, but that's like everything starts with that first step. I love that story. I kind of have-I personally kind of like a love/hate relationship with velocity. I do-you know, as you went on, I do think we actually see more eye to eye than I thought we were going to get you. So I'm glad that we're there. But a lot of people view that as the metric for a team but we really view it as like it's a data point of many other things of like when velocity was high for a sprint, that's a data point we're doing something well. We're not sure what it is that isn't success in itself, where we just gripping stories really well. Was it the type of story or are-did we just underestimate? Can we get better at estimating? And then sometimes when velocity’s low, is that an indication of a team morale problem? Or it could just be simple as people weren't feeling well and other factors. So, we use it as a data point to learn more about it, kind of where our team is, and then try to get that as an indicator to go investigate what actionable items are. And I think going to the story, I think we're my-my hate of velocity purely when you use it as the metric started was, I was at an organization that was just behind on their timeline, which happens. So this person had the idea, we're going to make a war room, we're going to put everyone in a room together they're going to sit there, they're going to work together, and they're just going to focus on this. And everyone has to be in by nine. No one leaves until five. And it was very regimented, and I thought “This is a nightmare. This is going to go horribly.” But the thing is, velocity did go up and after two sprints, I’m like, “Okay, wait, let's see then why did it go up? Because this is not a great working condition.” And we kind of came away with what we really solved there was the communication channel between devs and product managers was just slow and we kind of solved that problem. So, I thought, great, let's get people back to comfortable working environments and work on solving that. Like no, no, no, velocity is going up, staying with the war room. So over three months, people were in a room together working and-and to the shock of probably no one, people just started leaving. And then velocity dove again and then they were like,” Gosh, what went wrong? Velocity was going so well.” And, and so that's where I came away with, it's a data point that you look at to see and indicate it is not the goal because sometimes there's just so many consequences when you're just like laser focused just on that.
Jennie: [39:21-40:00] I mean, velocity is easy to game. So we actually-our assumption is that we will hit the sprint 50% of the time. It's great when we do more, but it's really used again as a data point to help product figure out what will be completed and released within a given two weeks. We also don't want to get to the end of the sprint and be releasing twenty different features all at once just to say that we did it. We definitely want to be a little more-a little more cautious there, and make sure that the quality is there along with some of the quantity.
Conor: [40:01-40:12] Yeah, I think if you're going to try to do a speed to value main goal metric, in my mind that it has to be cycle time Shankar, what kind of ways are you measuring that team performance at DataStax.
Shankar: [40:13-41:56] Actually we’re-truth be told a little bit early in in our journey there, we're just starting to make some strides. We're taking an approach which is a little bit more outcome-driven right? So, we're actually establishing a whole bunch of metrics that measure how well we're delivering to our customers and so we've got a whole bunch of customer experience kind of metrics that, you know, essentially our activity controls those-those-those metrics, right? It's whether it's, you know, performance of-of our database, it's availability of the database. It could be, you know, how easy is-is there is the database to use right? All of those kinds of outcome metrics. And we're really moving to where we want to have all the developers on the team kind of on and help drive these outcome metrics. And we want to measure impact based on how well those outcome metrics are moving rather than the input, which is, you know, the activity that that we do in terms of building things and so on. Right? Now, having said that we are looking to put measures in place that would look at the team's productivity. So, in other words, are there, you know, issues with tools, with the environment and so on that are getting in the way of folks being able to do things efficiently and productively, right? And we want to get rid of those for sure. But where we are headed is really to make everyone accountable for outcomes and really measure outcomes that we are delivering across the board. Right? And this is something we're doing in partnership with our PM team as well. And so, we're doing this together.
Conor: [41:57-42:05] This sounds like it's part of your kind of philosophy I've heard you talk about, bringing product thinking to engineering and driving that long term customer experience.
Shankar: [42:06-43:05] Absolutely. Absolutely. I think-I think that's when the magic happens is when, you know, every engineer can empathize with the needs of the customer and, you know, internalize and actually when they-because that's-that's where the most powerful tool is, right? When-when somebody puts a code in that code is ultimately the determinant of what experience somebody is going to have. And if-if-if the person writing the code can really think about the customer experience, then you get magic that you get amazing stuff. So-so that's kind of what we're striving to. And this is inspired by the time I spent at Amazon where it was one of those places where I saw, you know, every engineer-I've never seen this anywhere else, right, I would love to create this even a DataStax, but people debating with each other what was better for the customer. Right. This is, you know, junior level engineers, right? Having-having debates in the hallway, which is which is amazing, right? That that's kind of what I love to create.
Conor: [43:06-43:14] Ben, I've heard you talk about this concept, too, of building a team around product and people. Can you maybe expand on how you approach that at Stack Overflow?
Ben: [43:15-45:37] Yeah. I mean, as we said a couple of times, like the people on the team are kind of what's going to determine the success. And, you know, a lot of people, a lot of organizations will think of, well, we'll start a dev team. This is when a dev team is made of, this is how a dev team behaves, and this is what a dev team does. And then ignoring the individual people within that team. And even what they're trying to accomplish can vary wildly. And so we try to customize kind of how the team meets, works together or even communicates with each other based on some of their strengths and weaknesses and the personal preferences. They're-like just for an example, like some teams that I have, there are just a group of folks that just do very well when they're together working together. So, we facilitate stories to be written to be more palatable to peer programing. And even though we don't enforce any of that, it's none of this is hard. But like during grooming this thing, I would love to work on this with you. And once we started allowing them to work in this way, that was more natural to them, the team just started going faster and-and this is all done before it even gets to the grooming session or during the planning session of when we're writing and organizing this work, even back to when we're writing textbacks, like this is how we kind of organize the team. And because they're doing that kind of peer programing, the number of sinks that we have to do across the team can go down. They're already communicating a lot with each other to begin with. We've seen like in-less development problems and deployment problems. So just kind of customizing that works structuring the team for them know it paid off. And then even on the flip side, there are some people that they're just really focused on the review cycle of code. The unsung hero of-of any sprint are the people that go do the code reviews, give good feedback, and then still get their work done. These people, if we could give congressional medals or something, I would want it to be for that. That is great work. And there are some teams that this really depend on that for, for some of the code or the type of work that they have and trying to facilitate now-at time or even kind of sync up group sessions for us to review code together, kind of changing the team cadence to allow for that, you know, that pays off. So just the idea of trying to have like “This is what a dev team is, this is what they're made of, and this is how they meet.” It can be kind of narrow visioned looking at the people in your team, how they like to work and what they're working on and trying to adjust your team to that just makes for a much better recipe for success.
Jennie: [45:38-46:38] And I think that it's so important to allow your teams to kind of customize how they work best. We have a lot of more junior engineers, so software engineers and software engineer twos that have never really done code reviews and allowing them to essentially create a huddle, whether it's daily or a couple times a week, and have the principal engineer do a code review live with them to show them what they're looking for, how they're looking at the code. I mean, they all of our senior folks are able to give constructive, well thought out feedback and comments. But being able to show them the process that they use, they can then go back and keep that in the back of their mind as they're coding. So hopefully down the road they aren’t seeing as many comments and they're kind of proactively thinking about what could be caught in the code review process that can really just be taken care of right away.
[Music fades in]
Conor: [46:38-46:57] I love all the topics we touched on here today because I think starting from that metaphor we had of how do you build a successful engineering org, we've touched on a lot of key points of it. Fantastic. I love that we're getting all these actionable insights. And Jennie, Shankar, Ben, thank you so much for joining us at INTERACT today. I've really enjoyed this conversation and taken a lot away from it. I know our audience has too.
Shankar: [46:58-46:59] It was a pleasure.
Ben: [46:59- 47:00] Thanks very much it was great being here.
Shankar: [47:00] Yeah.
Conor: [47:02-47:06] Well, we'll definitely have to have you all back again soon and hope everyone has a great rest of your Interact.
Shankar: [47:07] Thank you.
Ben: [47:08] Thanks. Bye all.
Jennie: [47:09] Thanks.