#191 - State of Developer Experience 2024: Uncovering the Disconnect - Andrew Boyagi

[00:01:10] Introduction

Henry Suryawirawan: Hello, everyone. Welcome back to another new episode of the Tech Lead Journal podcast. Today, I have with me Andrew Boyagi, a DevOps Evangelist from Atlassian. So the reason I have Andrew in the show is because I saw the State of Developer Experience Report 2024, which was published kind of like a few weeks ago. So I find it really exciting and also some insights that I think will be interesting for us to discuss today. So Andrew, thank you so much for your time.

Andrew Boyagi: Hey Henry, thanks for having me.

[00:01:37] Career Journey

Henry Suryawirawan: Right. Andrew, I always love to ask my guest to first introduce himself or herself first. Maybe in your case, if you can mention any highlights or turning points that you think we all can learn from that will be great.

Andrew Boyagi: Sure. So hi, everyone. I’m Andrew Boyagi. I lead the DevOps evangelism team at Atlassian. I’ve led engineering teams for the past 20 odd years. And I really started my career in technical project management and was asked to lead teams very early on in my career. Uh, and really during this early part of my career where I was leading a lot of teams and projects, I learned a lot about what makes effective teams and how to get stuff to production at speed. Then through some org changes throughout my career, I accidentally ended up leading a service management team for around three and a half years. So I ended up on the ops side of the technology fence as there was a fence in those days. And during those three and a half years, I really learned a lot about service management and really the upside of technology. And by the end of this experience, I had what back then was a rare ability to speak both dev and ops, which is a valuable skill that still helps me even till today.

Immediately prior to joining Atlassian, I was an executive manager at the Commonwealth Bank, where I really worked at the intersection of dev and ops for many years. I led a large platform team there and together we were responsible for driving the adoption of developer tools across the organization. I had end-to-end responsibility for the SDLC over there. We built and rolled out an internal developer platform that was adopted by 7,000 engineers in 10 months. During my time there I also completed an MBA, which I found and I continue to find extremely valuable and highly recommend others to do.

I joined Atlassian around two years ago, where I lead the DevOps evangelism team, as I mentioned earlier. And I work, as a part of my role, I work really closely with product teams on the developer focused products within our company. And also work closely with customers on strategies to improve developer experience and productivity, as well as how to get started with platform engineering.

Henry Suryawirawan: Your story just now, right, we listened, I think it’s a perfect person to talk to about developer experience. You’ve been there from developers point of view, ops point of view, also led the team that built the internal platform teams. So I think it’s really great to have this conversation.

[00:04:05] State of Developer Experience Report

Henry Suryawirawan: So let’s go to the report itself, right? So State of Developer Experience report, which was published by Atlassian and DX, right? GetDX. So in the first place, maybe tell us a little bit of background. Why did you come up with this report and what the report is all about?

Andrew Boyagi: Yeah, I mean, there’s a significant amount of academic research that shows already that happy workers are productive workers. And really developers are no different to that. And at Atlassian, we’re on our own developer experience journey, and we wanted to get a fresh look at how people are thinking about developer experience, what’s contributing to a good experience, and what might be getting in the way of that. And really wanted a perspective from companies all across the globe on that. Really what we wanted to learn about was the state of developer experience across the world. What’s working and not working when it comes to developer experience. And obviously, we can’t leave AI out of the conversation. So we wanted to have a look at the future of developer experience when it comes to AI.

So as you mentioned, we partnered closely with DX, who, as many people know, provide a developer insights platform. And we really wanted to leverage their expertise for the developer part of the survey. In terms of how we constructed the survey, DX led the survey to developers, while our research partner Wakefield, and they’re a company that we’ve partnered with in the past, surveyed the engineering leaders. And that’s how we got the results that are within that survey.

[00:05:32] Developer Experience (DevEx)

Henry Suryawirawan: Yeah, I had a pleasure of having Abi Noda and few other GetDX people in the show before. So I think it will be interesting, just like what you mentioned, they cover the developer part where maybe your side is covering the engineering leaders, which I find quite refreshing in terms of survey. Let’s start from the definition of developer experience first, for people who may not be into this topic, although it’s getting hot these days. So maybe in your view, what is developer experience?

Andrew Boyagi: Yeah, the developer experience really focuses on the lived experience of developers and the points of friction that they encounter in their everyday work. And when we think about everyday work, it’s their job in taking something from idea stage into shipping software to production and maintaining that along the way. If I was going to boil that down, it’s really about how developers think and feel about the tools and the frameworks that they use to deliver technology on a daily basis.

[00:06:25] DevEx Across Companies & Teams

Henry Suryawirawan: Right. So I think as developers, definitely, we all have pain points and friction along the way, right? Especially to getting things done to the production. But is there any difference in terms of developer experience, you know, from different engineering teams or companies, maybe from startups, big companies, or maybe small teams, big teams? Do you see any kind of difference in terms of definition?

Andrew Boyagi: Yeah, I think it’s really important that we do align on the definition, because a lot of the time when I speak to people about developer experience, they’re all coming from different places. So I think we should hone in specifically that it’s about the friction they encounter during the course of doing their job, which is to ship high quality software to production. And then when you talk about the differences, maybe in different size companies and different types of companies, there’s no one size fits all developer experience. And developer experience is generally pretty unique to each company. The size of a company does play into developer experience in a few key areas.

So if we look at tools, for example, large companies generally restrict access to certain tools and focus on standardization, while smaller companies generally give freedom, which also has a downside because it can lead to tools sprawl. And for a developer, that might mean they have to make decisions about which tool they’re going to use, which can actually detract from a positive developer experience. Then if you look at processes, large companies are generally pretty process heavy and they’re focused a lot on governance and quality control. Whereas startups, on the other hand, are a lot more flexible. But they lack a lot of processes and frameworks which can also have challenges for people trying to deliver software.

And maybe the third angle that we can look at this through is the number of people. And this is not something that people talk about a lot, but the more people you need to interact with to get things done, generally the slower it takes to get those things done, which impacts experience as well. Now, the smaller the company, obviously, there’s less people to interact with, but that means there’s more that you need to do personally, which also can have its own impact. So like I was saying, it’s all unique. It’s all very dependent on the company and the industry that that company operates in, but there are some commonalities across companies when it comes to developer experience.

Henry Suryawirawan: Thanks for highlighting all these, right? So definitely like people, process, technology, right? It covers everything. And I think for some people who may have the misconception that developer experience in a smaller company is great, so probably you’re not right, right? Because there are so many other things like processes, you mentioned, standardization of tools, and also small number of people doesn’t mean that you are moving fast in a proper way, right? So I think that’s definitely a key thing about developer experience, like no one size fits all.

[00:09:20] Report Key Highlights

Henry Suryawirawan: So maybe let’s move on to the actual gist of the report itself. So can you mention the top highlights that you find from the report?

Andrew Boyagi: Yeah. So I think one of the key things that really stuck out to me when I was going through the report was that there’s this massive disconnect between engineering leaders and engineers themselves. So if you look at many of the results of the survey, they all talk about maybe what are the top frictions that appear within the survey. So if you look at what developers are saying, you know, their top five things that get in the way of a good developer experience is very different to maybe what engineering leaders are saying.

And really one of the highlights of that area is that the impact of AI and how people expect that to impact developer experience. So if you think about from the leader side, almost 100% of leaders were saying that they expect AI is the best way to improve developer experience within their organization. However, when you look at engineers and their response, they’re saying that, two out of three engineers, are saying that they’re not seeing significant gains from productivity just yet. Now that’s really interesting because it could have a few different impacts when it comes to developer experience. So if leaders believe that AI is the best way to improve developer experience, then they will invest heavily in AI. But it may not actually impact or change the developer experience for the developers.

So that disconnect between engineering leaders and engineers is quite prominent throughout the report. But if we take a look at maybe productivity and, you know, where developers and engineers think that that time loss is occurring, from the developer side, they believe that tech debt, inefficient documentation, and build processes are the top areas that are causing them to lose time or be less productive than they’d like to be. Then when you look at from the leader’s perspective, you’ve got understaffing, expansion of the developer role, and new technologies that they think is impacting their teams the most, being able to get things done.

So these are like quite broad areas where engineers and leaders are saying that they’re losing time. There’s a disconnect, which indicates that they’re probably not speaking to each other or not, maybe not too familiar with the challenges that their teams are having. But there is interestingly also a correlation between less obstacles and higher satisfaction with developer experience investment. And one explanation or one potential explanation for that is that companies who understand the challenges that teams face and prioritize fixing these end up with a higher satisfaction amongst their developer community than those who don’t prioritize and understand what the challenges are.

Henry Suryawirawan: Thanks for sharing all these highlights. So definitely really interesting about the disconnect. Although, I think in many companies, we do understand and realize about this disconnect, because leaders always think one way while engineers are feeling the different things, right? So definitely, it’s highlighted in this report again.

[00:12:41] AI Impact to DevEx

Henry Suryawirawan: And let’s start with AI, right? Since it’s like the trending topic these days. It’s very interesting that leaders find AI to be the solution to improve productivity, while engineers, like what you mentioned in the report, two out of three don’t find it significantly improving their productivity, right? Why do you think there’s this disconnect? And in terms of AI, what kind of things that you are referring to? Is it like code, like Copiloting, you know, or is there something else that you’re looking into categorizing as AI?

Andrew Boyagi: Yeah, so the survey just basically asked AI, didn’t go into the detail. But generally when you look at what’s available for developers in terms of AI across the market, they’re really focused in on the coding part of a developer’s role, which sounds logical. But there’s a few things that we should keep in mind. One is that developers, and I think an industry accepted statistic is that developers spend around 30 percent of their time coding. Now, a lot of the AI tools are aiming to optimize that 30 percent of their time. Now the thing is, that is the 30 percent of the time that they spend at work that they really enjoy. I don’t know about you. I know you speak to a lot of developers, as do I. None of them have ever said to me that they spend too much time coding. So whilst it’s awesome that, you know, AI can help with that part of the role, I think it’s a real missed opportunity to improve developer experience. And when you look at the experience part of a developer’s role, most of the complaints come from the 70 percent of the time of their role, not that 30 percent that they’re spending coding.

So I think that AI can improve developer experience if it’s targeted at solving problems that developers are currently facing. And it all comes back down to something that I like to say is if you want to understand where things can be improved, you need to ask your developers. There is no one else, no consultant, not me, no podcast you can listen to that’s going to tell you how to improve the developer experience in your company. Your developers can tell you. So if you ask your developers and they give you a list of things that they feel can be improved, AI is probably going to be a good solution for many of those things. But blindly picking an AI tool off the shelf and asking your developers to use it is probably not going to improve the experience for developers in your company. In fact, it might make it worse.

[00:15:13] How Developers Spent Their Time

Henry Suryawirawan: Thanks for highlighting about 30 percent of the time only spent for coding, right? So I think it’s really interesting. I think we all kind of like experience it. Since you have so many, so many experience in the industry with all these challenges as well, can you highlight what are some of the top things that developers spend time on apart from just coding?

Andrew Boyagi: Yeah, I mean, coding has always not been the extent of a developer’s role. Ever since I got into the industry, you know, there’s always that translation from what is this thing that we’re trying to achieve or the problem that’s trying to be solved? And coding really is the means to solve that problem. So maybe that’s something that’s worth highlighting before we get too much deeper. However, if you look at the evolution of a developer’s role over the last 10 or 15 years, you’d say that maybe most of their role was coding in the beginning. And slowly as the industry has evolved, a lot of things have come in.

So I remember a long time ago when I was working at an organization and they thought it would be a good idea for developers to start doing testing. Which is logical, but it meant that engineers now had to learn about testing and testing techniques and include that as a part of their role. Obviously, with the shift to DevOps, developers are now taking on a lot of operations responsibility as well. And it all makes sense, they’re all good things, but it is more that developers need to know and need to learn about.

You talk about cloud. There’s another set of technology, which is awesome. You know, cloud is, I’m a big advocate, obviously, for cloud. However, it is another thing that developers need to know how to use and to understand. And that’s all from the development side of their role. And there’s a lot of complexity that’s growing in that area, especially when you start talking about AI and ML and all these other things that are coming in.

Then there’s also complexity that comes from the company that you work in. So we spoke before about, you know, the difference between a large company and a small company with processes and the amount of people you need to interact with. There’s also complexity that comes from the industry that you work in. So if you work in a heavily regulated industry, there’s a whole another set of complexity that comes along with being a part of that industry and being a part of that company.

So when you look at how the developer role has expanded over the last 10 or 15 years, what started off as being able to go to uni and learning how to do something to now having to know about DevOps and testing, cloud, complexities, regulations. And all of that with a backdrop of companies want to get to production faster with higher quality, and they want to be first to market. There’s a lot of pressure and a lot of complexity that developers need to deal with these days.

Henry Suryawirawan: Right, definitely seems like a lot, right? So especially, we always tend to shift left everything now, yeah. So I think there are so many things that developers need to think of, be part of, implement and things like that. And so many technologies that these days we have to deal with, even though just to, you know, deliver a simple service, right? So I think I can really understand the pain of being the developer these days.

[00:18:21] How to Improve DevEx

Henry Suryawirawan: So speaking about all these dealing with complexity and, you know, developer satisfaction and all that, I know that the DX team also has published a research paper called DevEx before, right? So a little bit on the theoretical side. So I think they highlighted like three core aspects of developer experience from that research. Maybe if you can also relate your story just now with that particular theory, that would be great.

Andrew Boyagi: Of course. So I think like when you look at how to improve developer experience, there’s a whole lot of frameworks and there’s some really good material out there. I try to take a little bit more of a practical approach to improving developer experience. It’s something that a lot of companies ask me about on a regular basis. And to me, there’s a very simple process you can go on. The first thing is, like I said earlier, start by speaking with your developers, understand their pain points. I once spent two full weeks just asking developers in a company, how can the way software that is delivered here be improved? In those two weeks, I’ve got a backlog that would probably take three or four years to get through just in those two weeks. So if you ask your developers how things can be improved, they will definitely tell you. You won’t be able to stop them from telling you. So for me, the first step is always to ask your developers.

Now in a large organization, that’s not scalable. You can’t ask every developer what needs to be improved in a company. So start off with a group of maybe senior developers from across the organization, get a sense, and then move into surveys. So surveys are a great tool that are scalable. You can get a sense across engineering and really confirm or maybe debunk some of the theories that you had around what’s impacting developer experience across the company. By then you’ve got a pretty good data source. You’ve got some qualitative data to go off around what can be improved.

To me, that’s the point in time where you pick what you’re going to measure. So you find what areas require improvement and you find ways to measure those. Now we’re lucky and unlucky in technology in that everything can be measured. So it doesn’t mean you should measure everything, but whatever you want to measure, you probably can. So if you identify a list of priority areas that you think require some improvement, it’s generally pretty easy to track those over time and measure the improvement that you’re having.

And then it’s just about the feedback loop as you mentioned. So as soon as you identify what you need to measure and it’s been baselined, check back in with that group of developers and ask them, hey, we identified these things that need to be improved. Here is how we are planning on improving these things. What do you think? Get their feedback there. And then after you’ve implemented, circle back again. Did we actually improve the thing that we said that we would improve? And so to me, it’s a continuous cycle. It’s not something that’s one and done. You won’t just pick a set of measures. It won’t be the same problems for every company or every time, but to me, that’s just the basic process to go through if you’re trying to improve developer experience in your company.

[00:21:31] What to Ask Developers About DevEx

Henry Suryawirawan: Yeah, so I think for engineering leaders out there, right, this is, I think, the key, right? So don’t forget to speak to the developers. Sometimes we just want to measure things, you know, in the background. Or maybe using some tools, you know, implement the tools, I don’t know, either agents or counting number of something, right? So we have an abundance of metrics that we can measure. But I think speaking to the developers is still the most important thing. And I think in terms of speaking to the developers, right, so do you think that we need to have a set of a few areas that we have to ask them, or is it more like a free conversation and they just tell their problems, you know, freely. Or is there any, some kind of, I don’t know, like framework or set of questions that the leaders should ask developer?

Andrew Boyagi: I like to start really broad. So personally I ask how can we improve developer experience here? In my experience, when I did that, I did that for two weeks full time, I got quite a surprising response from some teams. So I expected to hear a lot about the tools, as you mentioned, and tools being slow or the wrong tools, which are definitely favorite topics for developers. And I did get that. But I also got quite a lot of feedback on the processes that the company tries to implement in terms of quality and governance. And that might not sound like a surprise, but that these processes were impacting developers in ways that were unexpected.

So for example, you know, there was a lead time before you could do things in production, which obviously sounds like it’s going to slow things down. But the way that that increased cognitive load for developers and what they had to do within those two weeks, was a little surprising and it was quite varied across the organization. So if you ask a really specific set of questions, you probably won’t get that level of detail, which I think you need when you’re starting.

So like I said, it’s not scalable, but I would definitely start with free flowing conversation. And ask the developers to frame it in terms of, because you don’t want to just to hear a list of complaints. You want to hear something constructive as well. Complaining is okay to an extent, but you really need to understand the impact of what they’re saying. So I’ll start with how can we improve experience here or how we develop software here. And then ask them what’s the impact of what you’re saying, how does this impact you and just guide them towards explaining that to you so you can better understand the problem.

Henry Suryawirawan: Yeah, I’ve been in this situation before as well, like speaking to developers, you know, they share their challenges in a open hearted way, right? But taking actions or maybe like the gist out of those conversations, especially if it’s like an open conversation, it’s really hard for you, right? So I think getting some, I don’t know, like tools like DX will definitely help in order to come up with the, I don’t know, a few areas. Also how to quantify the complaints from the developers. I think maybe asking questions seems easy, right? But actually coming up insights from those answers probably is also something else.

[00:24:22] Impact of DevEx on Developers Retention & Attraction

Henry Suryawirawan: So we spoke about developers when they have less, you know, friction in the developer experience, they seem to be more happy, satisfied. And maybe that also relates to the retention or maybe attracting talents, right? Can you speak a little bit on this? Like what do you see in the industry? Are people really moving away from companies who have really bad developer experience? Or is there some kind of a attraction, if let’s say, you find a company that has good developer experience?

Andrew Boyagi: Yeah, absolutely. I mean, like if you look at within our survey that we sent out, 63% of developers were saying that developer experience is important when deciding to stay in the current job. Now, a lot of the time when I speak to companies about the impact of developer experience on recruitment, they generally think about it in terms of attracting new staff. They very rarely are thinking about it in terms of retaining the staff and the good staff that they already have. And if you dig deep into that, think about the cost of retain or not retaining your staff. It’s costly in many ways. So there’s the cost of hiring and onboarding new engineers. There’s a huge productivity hit across existing staff. So if someone leaves, the existing staff have to take on that workload in many cases. And then they also need to upskill new people when they join the company. And then there’s the loss of company knowledge which is very expensive when it comes to software development. So I’d say that in terms of retaining staff, it’s very important. Developer experience is very important in that regard.

When it comes to attracting new talent, absolutely! Engineers quite often ask about the tech stack and the tooling that’s used when they’re looking to join another company. Companies also have reputations for their developer experience. So, you know, it’s critical when it comes to attracting and retaining the talent that you have.

Henry Suryawirawan: Yeah, you mentioned about reputation. So I think maybe if it’s not published publicly, or maybe the marketing side of the company, people always speak about this in various forums or maybe on social media or whatever that is, right? So I think that’s a really interesting, right, to hear that in the industry, maybe they have classified certain companies with bad developer experience, good developer experience and things like that. So definitely it’s also one thing for a company to invest in improving their developer experience so that they can continue retaining the talent, the first thing. And also attract new good talents as well to the company.

[00:26:50] The Danger of Traditional DevEx Measurement

Henry Suryawirawan: So I think we mentioned about measurement and all that, right? So I think let’s just go there a little bit, because many leaders are still kind of like misled by all these metrics. Things like, for example, the velocity of your sprint, I don’t know, number of test coverage or things like that. Maybe, do you think this is still pretty much relevant or should we stop doing all this altogether? And what’s your take about all this traditional measurement?

Andrew Boyagi: Yeah, I mean, let’s take a look at the survey results. The top five ways that leaders say they’re measuring productivity is through the amount of code written, deployment frequencies, story points per sprint, hours worked, and the change failure rate. Now, personally, I would not use any of those metrics when it came to productivity. And the funny thing is, the leaders in the survey, more than 50% of those leaders said that those measures were actually not effective measures to capture productivity. Which leads to the question, why are they using them? And in the conversations I have, it seems to be that two main reasons why people are using these metrics. Number one, they’re easy metrics to get. They’re available. You can get them quite easily. And so it’s a metric. And B, or number two, they don’t know what else to use.

Now, almost every senior technical leader I speak with is asking for advice on measuring developer productivity. And I get why. You know, they’re accountable to maximize the investment in technology. I agree that it’s a core function of a leader to make sure that their team is working optimally. It’s a core responsibility. But asking how to measure developer productivity, in my opinion, is the wrong question to be asking. We should be asking how can we help our developers be more productive? The more effort and time you spend trying to find the right measure for productivity is time that you’re actually taking away from helping developers actually be more productive.

Now, a lot of the time when I speak about these things, people misinterpret what I’m saying and they think I’m saying don’t measure productivity. I’m not saying that. Productivity is very hard to measure and this is true for all knowledge workers. It’s not just for software developers. I think it’s more of an issue for software developers, because we have a lot of outputs that can be measured, and then people try and use these as proxies for productivity. There are some companies who actually measure productivity quite well for the developers. If you look at, or you listen to any of these stories, there’s one thing that all of these companies have in common, and that is that they all measure productivity differently. And the reason for that is what goes into productivity is quite unique for every organization.

So to me, the two inputs to productivity are developer experience and engineering culture, which, again, is not something that’s spoken about frequently enough. And really, those two things are extremely unique for every organization. I can tell you that from an Atlassian perspective, when I joined Atlassian, I was pleasantly surprised actually with the culture that we have here. We have an awesome culture internally within Atlassian. There is no way any other company could replicate that culture. It’s impossible. Even if you took all the tools and leadership principles and all these things, and you put it into another company, the culture is going to be completely different. It’s not something that can be controlled so easily.

And so if the experience and engineering culture are so unique, and these are core inputs to productivity, then the way you measure productivity has to also be unique to your organization. And so again, I’ll go back to that journey that I mentioned earlier, and that would be to find out what are the areas where you need to improve or improve as a company. And I would measure those areas in terms of improvement.

Henry Suryawirawan: Right, so I think you mentioned top metrics that leaders use, so things like, for example, story points, deployment frequency, amount of code, which is, uh, I think still pretty surprising. Because we all know that counting code probably is not the right way. And people can game these things, right? So if they know what is being measured, they’ll just try to optimize those metrics instead of actually improving the underlying thing.

[00:31:15] Importance of Engineering Culture

Henry Suryawirawan: And you mentioned something about culture, right, which is very unique. Thanks for highlighting that. So in your view, I know this is probably hard to actually kind of like quantify what is culture, right? So what makes a good engineering culture? Maybe from your experience seeing different companies or seeing different development teams.

Andrew Boyagi: Yeah to me, a healthy engineering culture… Sorry, firstly, what goes into an engineering culture? There are many things that go into culture. Things like organizational structures, how decisions are made, how people interact with each other, what’s encouraged. You know, there’s stories that live within an organization. Stories that become legend. All of these things feed into what’s an engineering culture. But to me, really a healthy engineering culture is a culture that encourages continuous learning and continuous improvement.

One way that this is done within Atlassian, for example, is we have a ritual which is called CheckOps. So within our developer experience platform, which is named Compass, every engineering team does a, it’s kind of like a retrospective. Now, the problem with the traditional retrospective is you get a facilitator or someone from the team to come in and say, all right, what was good in the last two weeks? What could have been better? What can we do to improve? And everyone sits there and tries to remember, oh man, what’s happened in the last week or two? I don’t even know what’s happened. And then you just come out with whatever is top of mind for the team at that time.

What I like about CheckOps and how we’ve embedded this within our platforms is it brings together all the data from the software components that the teams are responsible for. So they can see all of the events, like what incidents have they had. They can see their scorecards, you know, how are we going on application readiness and security? Are we failing anything? And so they’re presented with all of this data on one screen, and then they’re able to say what was good, you know, what wins have we had? What should have been better? What could have been better? And then what actions are we going to take to improve?

Now, these things go straight into their backlogs, obviously, as improvement items. And this way, they’re always able to manage that investment in time between innovation and between managing tech debt, and they’re always learning. So, as you know, and how teams progress, teams learn over time and they learn together. And so I think really an engineering culture where teams are encouraged to learn and encouraged to improve and encouraged to experiment is really that healthy engineering culture that companies are looking for.

Henry Suryawirawan: Yeah, I think it boils down to this continuous learning, continuous improvement culture, and also psychological safety in a way, right? Because for teams to thrive, actually, they need to feel safe. They need to be able to say, you know, sometimes we make failures and, you know, incidents happening and all that. And try to improve from that experience. So I think definitely that’s a really key thing, right, in terms of engineering culture. And I actually like what you mentioned about legendary stories, right? Typically it’s like the mistakes or the errors that happen within the team, right? So I think you can actually also gauge a good culture just from hearing that legendary stories, right?

[00:34:24] DevEx Frameworks

Henry Suryawirawan: So I think, talking about improvements and, you know, learning, sometimes people take it from what’s out there. Things like, for example, common frameworks that people mention, you know, like the DORA, you know, the DevEx, and also SPACE. These three frameworks probably are like the top. Do you think it’s also a good thing we follow those frameworks and we learn from there? Or what’s your take about all these?

Andrew Boyagi: Yeah, they they are fantastic frameworks, um, when it comes to developer experience. But like any framework, you never just pick it up and plop it into your company. You take a look at it, you learn from it, you adapt it to your specific situation. And I think you’ll find that the authors of those frameworks heavily encourage you to do that. They’re more about how do you think about developer experience? How might you measure it? You know, what are some examples, which is perfect. That’s exactly what you want from a framework. I think the danger is when people try to turn those into a one size fits all and say, you can generically apply this to any company and it will improve developer experience. That is far from the truth.

So for example, you know, cause every company has a different focus and different challenges and what you measure will be different. So for example, within Atlassian, we went through that same process and we found that quality documentation was something that was troubling our developers. So now we track how our engineers feel about the quality of documentation that they have access to within our organization, because that’s important to us. And that is something that our developers are telling is important to them. And so that’s something that we track. That’s probably not going to be something directly called out in those frameworks, but it will fit into the categories that the frameworks outline. And so that’s just one example of how you can take a framework, and these are fantastic frameworks, you should definitely leverage them, but apply them to your specific scenario.

Henry Suryawirawan: Yeah, I think that’s an important point, right? Because many leaders, actually, I had also a conversation with Nathen Harvey, like back then, right? So he mentioned, like for those leaders who read the report, they just skip all things and just come up to a particular page where it mentioned the four key metrics, right? And, you know, just implement that to the team. So I think the most important thing about framework, also understanding where they are coming from, what’s the definition of stuff, and why they come up with the four key metrics, for example. So I think definitely leaders who listen to this, don’t just take whatever frameworks out there and just implement, right? Adapt from your context, right? And also just take the best things that you can use in order to improve your developer experience.

[00:37:02] Platform Engineering

Henry Suryawirawan: Another thing that people mention in terms of developer experience is always about platform. You know, building platforms, internal tools, platform engineering teams, cloud platform, whatever platform. What’s your take about all these platform? Is it really the kind of like go to solution for improving developer experience?

Andrew Boyagi: I think if you look at the promise of DevOps, I think teams who or companies who want to be successful with DevOps need a platform. And, you know, we spoke earlier about how the role of the developer has changed. You know, they need to do more. They need to know more. They’re spending a lot of time learning and doing things that are outside of their core mission of shipping software to production. It’ll come to a point where every team hits the complexity limit where the complexity just gets on top of them and they’re actually spending more time managing complexity than actually doing their job.

And so a platform, the intention of a platform or an internal developer platform, and I mentioned earlier within Atlassian, our one is called Compass. It really aims to reduce that cognitive load on developers and abstract away some of the complexity associated with being a developer. The complexity never goes away. And it’s an important thing. Companies always ask me about how can we reduce the complexity that our developers need to deal with? You can’t, actually. The complexity does not go away. It still lives there. A good platform will abstract that complexity away from developers, but somebody still needs to deal with that complexity.

So if you look at cloud, for example within our experience platform Compass, we have templates which allow you to, you know, through a few clicks, defining some parameters, you will spin up whatever infrastructure you need in the cloud. You can spin up your CI/CD pipelines. And developers can, you know, quickly get an environment stood up and start focus on their core role. So a good platform will do that. However, that complexity is still there. It’s just abstracted by the platform team and the platform to manage.

And so I do think that to be successful with DevOps and, you know, to manage all the complexity, both that that’s intrinsic to a developer and organizational complexity that we mentioned earlier, a good platform is the solution to helping developers have a better experience at work and also be more productive.

[00:39:29] Platform Buy vs Build

Henry Suryawirawan: Yeah. So speaking about platforms, there are definitely different types of platforms out there. You mentioned cloud platform, the CI/CD and all that, right? So in your view, in terms of investing time, what kind of platforms should we use? Probably, sometimes also, I assume that we don’t always build, right, but we adapt from what is existing as well. And also maybe improve, customize a little bit. So what do you think are areas that we should think about in terms of platform, platformizing it, if that’s the right word?

Andrew Boyagi: I mean, for us internally, we, we, I was referencing our internal developer platform, which is Compass, which like you said, a good platform aims to reduce cognitive load. It aims to improve or promote a healthy engineering culture. And it’s also heavily extendable and customizable. Now there are a lot of companies out there who sell platforms, and with the promise that they’re going to solve, you know, whatever developer experience problems you’re having or they’re going to turn things around for your developers. No company can do this for you. However, they can skip you straight to the level of solving the problems that your particular company is having.

So for example, if you…, and I have built a platform in the past in, as we mentioned earlier. It was a long process to design and to build and it required heavy investment. And at the end, we got to the level where of the products that you can currently buy off the shelf. So for example, Compass, what you can get in Compass is already more advanced than what we invested in and what we built ourselves in my previous company. So by buying a platform, it’ll get you most of the way there and you’re able to then extend and customize that platform to solve the unique challenges that your company is having that no one else is probably having.

So I think it’s more about, it’s not build or buy, I think it’s buy and build. So you probably buy a base platform and then build on top of that so that you can solve the challenges that your particular company is having.

Henry Suryawirawan: Yeah. Speaking about platform engineering, I think I spoke also to Gregor Hohpe in previous episode. I think one good definition that he mentioned about platform is like platform needs to enable you to do more, right? It’s not like constraining and making people’s life difficult, right? I don’t know, like to deploy something, to create infrastructure. It has to be something that we can build on top and customize and make everyone much more productive. And actually, also very important that platform that can support something like a self service thing, like for developers to actually just do it themselves.

[00:42:03] Self Service & Reducing Wait Time

Henry Suryawirawan: Think of like cloud platform, they all have this console, nice console that you can just go click here, click there, and then the things just spinned up, then you can start using it. I think that’s also another good angle in terms of building platform. Yes, don’t just think of building platform as something that is constraining, standardizing things, and probably also make life difficult at the end of the day for developers, so.

Andrew Boyagi: Yeah, I mean, if you look at what are the things that developers commonly complain about when it comes to developer experience, and one of those things is wait time. And it’s waiting on another team, waiting for information, looking for information. These are all common themes that developers raise. And so, moving to a platform with a self service data model is something that really helps improve developer experience.

For Atlassian, when Atlassian started the microservices journey that it continues to go on, there was a problem around who owns a particular service, what does it do, where does the documentation live. And these things lived on multiple different pages and people were spending a lot of time looking for information. Which is why they actually developed Compass in the first place. It has a component catalogue which really stores all the metadata around a software component, so that people don’t need to go looking for things. They don’t need to wait for another team. Everyone in the company knows which URL to go to, to find exactly what they need. And they self serve most of the information. And by implementing this type of model, what you actually get to is something that I like to call is a high collaboration model, high value collaboration.

So if you think about low value collaboration, that’s one team reaching out to another, asking them, hey, where’s the documentation for this thing? So both teams are taken out of flow. The first team is taken out of flow because they have to wait for the other team to respond. The second team has to be taken out of their flow so they can respond to the first team. And really, nobody gets any value out of that. It’s really just the beginning of their interaction. When you think about self service and high value collaboration, teams self serve nearly everything that they need. And so that when they get to the point of actually engaging each other, both teams are likely going to collaborate and get an outcome that’s contributing to each of their goals rather than just answering basic questions about services and things like that.

Henry Suryawirawan: Yeah, I think that’s a really good point, right? The wait time, sometimes it happens, but we don’t count it, right? So it happens with many things, you know, collaboration, documentation, understanding certain requirements, what their ask, understanding certain API, for example. And also deployment wait time or build or deployment wait times. Sometimes, I think it’s also a classic thing that happens in many development teams.

[00:44:50] AI for Improving Documentation

Henry Suryawirawan: So speaking about all this wait time, I think one good thing that potentially AI can solve is actually to solve all this kind of problem, right? So we know about code generation part. But do you see some kind of a unique use cases where AI is being embedded into the platform such that we can solve some of this problem? Do you see some promise there?

Andrew Boyagi: Yeah, for sure. I mean, if you think about the component catalog I just mentioned that lives within Compass, really what it’s doing is structuring all the metadata that lives around a software component and that’s something that engineers regularly search for. And what does AI need to be successful? It needs a structured set of data in order to be able to serve the responses that we’re looking for. So I think setting up a component catalog and setting up your platform is really a good investment for the future capabilities of AI. But if we look at AI more generally, if we take a step outside of the common code gen and the coding aspect of a developer’s role, I think there is such good opportunities for AI when it comes to developer experience.

And my favorite example is actually writing documentation. So if you look at multiple surveys, almost any industry survey you look at, and you look at, you know, what are the complaints of developers, the number one complaint is poor quality documentation. Now, if anyone’s ever led an engineering team, the most common complaint they probably get is about writing documentation. So developers don’t want to write it. They want it, though. And so to me, that’s a perfect use case for AI. That’s, you know, we have a problem. AI is really, I want to say really good, but it’s getting better at writing documentation. And so that seems like a prime use case that can have a pretty nice impact on improving developer experience.

Henry Suryawirawan: Yeah, I think it’s a kind of like paradox, right? Everyone wants it, but not everyone wants to do it, right? Writing the documentation is always kind of like dreaded sometimes. And especially we have to read somebody else’s thing, right? Either code or artifacts, right? And try to understand what it means, right? So I think that’s also kind of like synthesizing things which is AI is good at. And you mentioned about having a lot of metadata that AI can be trained for, I guess, right? So I think like a chatbot to ask certain things about certain teams or certain APIs, I think that will be great as well. So I hope to see more such tools or such platforms, embedding AI into the platform such that we can leverage on this without having to wait for a long time, right? So I think definitely there’s a promise there. I haven’t seen, you know, a good platform yet that has all these elements, but I’m sure we can be confident that one day we can have all this.

[00:47:29] Feedback Loop for Improving DevEx

Henry Suryawirawan: So I think we have talked through so many different things. What other things that we haven’t covered that you think we can also mention here about the developer experience?

Andrew Boyagi: Um, I think we talked through the main points of the survey already. And so, you know, I’ll just summarize by saying the process to improving developer experience is generally the same kind of framework. This survey does a really good report of showing what are the general things that developers are saying are blocking their time. And they are general, because they’re not from one organization. And any developer who reads this survey, they’re not going to be surprised about the things that they say are blocking their time. Everyone who I’ve spoken to says, oh, these are things that are blocking me as well. However, it’s important to note that these are generic and they change from company to company.

And so if you really want to improve developer experience, start off by asking your developers. That is probably the key part of improving developer experience, is really about understanding what can be improved and then go on your process of improving those things. Implementing feedback loops, you know, just like you do with any product. And most companies today have products that they sell externally. And so it’s the same thing. You release a product. You get feedback on it. You improve it over time. It’s something that people expect to happen. And to me, developer experience is no different to that. Get the feedback and continuously improve it as you go.

[00:49:01] Atlassian DevEx Journey

Henry Suryawirawan: So you are working now at Atlassian. So maybe if you can share a little bit, what are some of the developer experience challenges that Atlassian is having at this point in time? And you know, what the team is trying to solve that, you know, maybe with some platforms or maybe with some tools that we all can learn from that journey as well.

Andrew Boyagi: Yeah. So, I mentioned earlier that we have our developer experience platform Compass. And that’s our internal developer platform. So, you know, Atlassian is very similar to many other companies who are on the developer experience journey. Atlassian went on the microservices journey. Uh, we’re a SaaS company. And so we offer a lot of our products in the cloud. We have a huge engineering base as well, a lot of developers and we’re growing rapidly. And so our challenges are quite similar to other challenges from other tech companies that you hear.

But we really focus on driving developer joy and through that initiative, we’ve seen a really great improvement in terms of developer experience. And so our developer satisfaction scores have gone up by 25% since we started that initiative, which is great. And, you know, we’re not stopping there. We’re continuously improving and seeking feedback from our developers and changing things as we go. But a lot of that has to do with our platform as well and how we try and reduce cognitive load, how we understand what the challenges are from our developers and how we systematically improve those.

[00:50:28] Importance of Transparency

Andrew Boyagi: One thing that’ll mention that we haven’t touched on just yet is around transparency. Now, what stops people from responding to developer surveys is when they continuously respond to developer surveys and they have the perception that the information isn’t doing anything or it’s not going anywhere. And so transparency is really a key part of improving developer experience. So getting that information from developers, relaying to them here is what we heard from you, continuously communicating with them, here’s what we’re doing around those challenges that you told us that we’re having, and then rebaselining and sharing the results as you go.

And then you know, that’s something that we do in Atlassian and something that I heavily encourage other companies to also do is really provide transparency around what you’re doing. Even if you’re not doing anything, it’s important to say like, you know, hey you gave us 50 problems that we need to work on improving. We’re only going to work on the first five, because they’re the most impactful. You know, I think as long as you give people that transparency and you set expectations, then that in itself can drive an improvement in developer satisfaction.

Henry Suryawirawan: That’s a really great point, right? Because sometimes with all these survey, it can lead to a survey fatigue, right? Where you just ask, ask, ask, ask. You have all the answers, but you don’t take any actions. You don’t share what actions are being taken. And you also don’t share what kind of impact from those actions that you have taken. So definitely like transparency, communication is still also important, especially from leaders to the developers.

[00:52:01] 3 Tech Lead Wisdom

Henry Suryawirawan: So Andrew, I really love this conversation. So thank you so much for sharing about all this developer experience and the report, which I find really fascinating. Unfortunately, we need to wrap up. But before I let you go, I have one last question that I always love to ask my guests to share, which is, I call this the three technical leadership wisdom. You can think of them just like an advice. So maybe if you can share your version.

Andrew Boyagi: Yeah, okay, three things. First thing is leadership is a craft that you need to learn. And being a good engineer or product manager or whatever your craft you came from, this will not make you a good leader. You need to learn what it means to be a good leader and build your skill just like you did when you became an engineer or product manager or whatever you did. And a part of that might be getting a non-engineering leadership buddy to help you along on your journey.

The second one is, and this is how I look at things. Culture is more important than anything else. And teams are more important than individuals. And to explain that, culture is everything. It’s not all up to the leader. But as a leader, you have a huge influence over culture, and this is something that you should intentionally design with your team. And in terms of teams over individuals, behind every great achievement is actually a great team. And to have a great team, you need to prioritize the team and not individuals. And you do this in the way you recognize effort, the way you celebrate achievements and the type of behavior that you accept from within the team.

And then last one is leadership style is important and you need to be intentional about your style. For me, personally, authentic leadership is what works the best for me and with me, as an employee as well. And essentially authentic leadership is about being genuine and being honest with your team. And if you do those two things, you’ll create a great environment where people can thrive.

Henry Suryawirawan: Wow, really awesome advice there! So thank you so much for sharing them. So Andrew, if people love this conversation, they want to ask more questions or they want to check out something. Is there a place where they can find you online?

Andrew Boyagi: Yes, I’m very active at LinkedIn. If you just search for Andrew Boyagi, I’m the only one in the world, so you will find me.

Henry Suryawirawan: Right. So I’ll put it in the show notes as well. So thank you so much, Andrew. So I hope people learn a lot about developer experience and how to improve it within their team and company. So thanks for that.

Andrew Boyagi: Nice talking to you. Thanks, Henry.

– End –