#182 - Building a Quality-Driven Culture: Enhancing Quality Practices Using QPAM - Janet Gregory & Selena Delesie

[00:01:46] Introduction

Henry Suryawirawan: Hello, everyone. Welcome back to another new episode of the Tech Lead Journal podcast. Today, I have one repeat guest and one new guest, Janet Gregory and Selena Delesie. So they just recently published a book, which I find very interesting for us to discuss and hopefully we can share some of the learnings from the new books so that you can improve your quality practices. So Janet, Selena, welcome to the show.

Janet Gregory: Thank-you very much.

Selena Delesie: Glad to be here.

[00:02:10] Career Journey

Henry Suryawirawan: All right. So since Janet has appeared in the episode before, so maybe this time I would like to ask Selena to share any kind of highlights or turning points from your career journey so that we can learn from you.

Selena Delesie: Well, I started my career as a software tester. Interestingly back in the day when I started, I did not know that was a real job despite going to school for computer programming. Yeah, so I quickly moved into managing and leading projects and teams. And one of the things that I enjoyed most was kind of bucking the trend of testers sat in their corner and developers sat in their corner and rarely did we ever cross. And I was, at least in the company I was in, I was one of the early folks to try to bridge that chasm and bring the programmers and testers together to work towards finding solutions together. And that worked really, really well.

And for the projects that I worked on with the testers and the programmers, we were doing agile ways of working without any formal agile awareness. Like, we didn’t know it was a thing at the time, because this was in the early noughts. And then into the next organization I worked in, I helped to bring agile informally into the organization and said, oh my gosh, this was a lot of what we were doing. Focusing on collaboration and communication and testing ideas before we wrote the code. And we just found that it was really great for everyone who was involved in the project. And we were more successful with the projects that we were working on as well that way.

From there, I shifted into doing coaching and consulting around quality practices as well as agile, and I’ve shifted into doing also executive coaching. So there’s a myriad of things, but the premise of all of the work that I do is really about focusing on the quality of the environment and the quality of the people. I do have the technical background to back up what I’m talking about and be able to speak with the technical people I work with, even though I don’t focus on that exclusively anymore.

But I found that when we focus on creating environments that are sustainable and healthy for people, they’re able to do amazing work. And all of the other benefits and gains that the organization is looking for just follows suit. And that said, having some of the technical experience allows me to speak with really technical people who think maybe I’m too fluffy for them. So, you know, there’s a bridge of being able to have all of the social, cultural, as well as the technical, you know, experience to kind of bring it into a package. It really goes a long way for everybody.

Henry Suryawirawan: Thanks for sharing your story. I think hearing just what you said, it seems like quite reflected in the book, right, the gist of the book. So talking about quality practices and how the social part and also the technical part coming in together into the team.

[00:04:58] Quality Culture

Henry Suryawirawan: So maybe let’s move to the book itself, right? The book is titled “Assessing Quality Assessments Using QPAM”. But the first thing that I want to talk about is about the quality culture. So, in your book, you basically mentioned all these assessments is basically to assess our team’s quality culture. But in the first place, what is actually quality culture? Like how can we define it? That’s probably the first question.

Janet Gregory: It’s funny because I’ve done a whole presentation on culture and keeping an agile culture as such. But when you start to think about what it means, it means the norms. How do people act? Do they do what they say? You know, walk the talk kind of thing. So culture is about how an organization works. It’s their ecosystem as such. When you’re assessing a team, does that team fit into that organization or does it have its own norms? Does it have its own rituals? Selena, I’m sure you can add to that.

Selena Delesie: Yeah, I think just in terms of that quality aspect of culture, to enhance what Janet was saying, it’s how do we bring mindfulness and insight and reflection into the work that we’re doing, whether it’s writing code or writing a test, or understanding our requirements, or just in how we’re communicating with the people around us, right. And having that idea of I want this to be a quality interaction or I want this to be a quality test and being thoughtful about that really makes a big difference, it seems. The organization, the teams, the customers, a lot of problems down the road, right? We’re building a quality product because we have a quality culture.

[00:06:42] Quality & Testing

Henry Suryawirawan: I like the phrase that you use, right? The mindfulness and the reflection of how you conduct stuff, right? I think many people associate quality culture with testing culture. Is this the right thing to think about or do you have any different perspective here?

Janet Gregory: The way I look at it is testing is a subset of the activities you would do trying to build in that culture. I know in our book we talk about 10 different aspects of quality. Culture is one and it’s one of the top ones. It’s a social aspect. But how an organization, say, communicates, that’s part of the culture as well. It’s a top down or bottom up? Does it do some of both? How does it communicate? So testing is really such a small part of the overall culture or the overall thinking about a quality culture. You have to have a good testing practice, but it’s just a really tiny part of it. Well, maybe not tiny, but…

Selena Delesie: Yeah. And with testing, it’s not just like you’re actually testing the software, testing the code, right? It’s we’re testing ideas. That’s basically what it comes down to all the way through, right? And when we can bring those questions early on, we’re helping to build that quality in as well. There’s so much more that goes into that. And you kind of touched on that, the mindfulness phrase that I use. I don’t believe that a lot of people would look at it that way. Probably not, but I mean, that is kind of like the underlying thing that I’m watching for, if there’s thoughtfulness that’s going into the work that people are doing, regardless of the particular task that they’re doing.

Henry Suryawirawan: Yeah, probably it’s the unconsciousness, right? Like people don’t have this mindfulness thinking about the quality practices that they have. And they always associate like with testing, testing, automated testing, right? How much coverage do we have? How many testers do we have? How many test cases?

[00:08:37] Quality Assessment

Henry Suryawirawan: So I think the point in your book that you try to bring is actually testing is just one part of the quality thing, right? And there are many other aspects. And I think maybe in the first place, what made you wanted to write this book? So like do-you find it difficult to actually assess people’s quality practices? Like maybe what made you write this book?

Selena Delesie: We’ve both been asked to do assessments for organizations in our past. And Janet was asked by a particular client to I think just better formalize what that might look like. Um, we came across a couple of other models and Janet said, hey, would you like to help me build something out? And then we suddenly had our quality practices assessment model and said, well, now what? We think we have something of value to share with the industry. So we hummed and hawed and tried out some ideas and landed on, you know, the easiest thing to do, which actually turned out to be more work than we thought, was to write a book about it so that we can get it out and share it with, you know, the industry, with our colleagues, so that we weren’t a bottleneck, that people could just pick it up and run with it as they want it to.

Henry Suryawirawan: But is it common that many people poorly conducting their culture, or, you know, like withholding their quality? And hence, I think, should every team try to think about doing this kind of assessment to actually know where they’re at? And actually how to improve.

Janet Gregory: I would like to see every team do an assessment like this. There’s lots of them out there, but if they know where they’re at, then they can start moving forward, right? If you never stop and assess whatever level you want to be at or whatever you could assess a small portion of what you’re doing, for example, and say, where are we? And if you never stop and look at that, then sometimes teams just start spinning their wheels and get in a rut and really don’t know what. They try fixing that and fixing that, but they don’t understand why or where they’re trying to even go for.

Henry Suryawirawan: Yeah. So in the past, I’ve tried my own assessment, so my own version of assessment. And actually, it is really hard. First is to define the model, right, or like the questions to ask. And then after that you probably have different kind of stages to actually represent where team at. I think it’s a really hard kind of problem. So I really thank you for coming up with this kind of assessment which I think is quite holistic.

[00:11:00] 10 Quality Aspects

Henry Suryawirawan: So maybe let’s move to the actually QPAM itself, right, the Quality Practices Assessment Model. So there are 10 practices that you include in this model, and also four kind of stages for people to actually know where they’re at. So maybe let’s start with all the 10 different practices first, like in the high level, what are those? And yeah, maybe we can talk from there.

Selena Delesie: Yeah. So, we’ll call the top level “quality aspects”, and then within each of the aspects, we have a number of practices identified. So the quality aspects that we’ve identified, the ten are feedback loops and culture and learning and improvement. Those are the social ones. Development approach, quality and test ownership are the social technical. And testing breadth, code quality and technical data, automation and tools, deployment pipelines and defect management are the non-social. So those are the technical ones.

So there’s a lot in there and I think a lot of people would be surprised that like, we did a lot of work and we reached out to colleagues to kind of prioritize them in terms of what are the most important ones to focus on first to get your organization in better shape. And I think a lot of people would be surprised that we don’t focus on the technical ones first, that we’re focusing on the social ones first. We start to get feedback loops in place early on, for example, like within our tests, within our code, our feedback loops within our team, feedback loops with our stakeholders, feedback loops with our customers. It really just helps us smooth everything else out for the other aspects down the road. What would you add to that Janet?

Janet Gregory: Yeah. How do we communicate those feedback loops is really about how do we communicate with each other. And if you don’t have those, then you end up with throwing it over the wall in a lot of silos. So, that was why we kind of chose that number one.

Selena Delesie: And it’s interesting, I often see in organization, because I do consulting and coaching working companies, I often see that the feedback loops is the first one to get cut when they’re feeling pressured for time or they’re too busy. So trying to focus on getting that in healthy shape, I don’t think that’s a compromise that’s worth making. And that alone, focusing on that and doing that really well is going to help reduce costs, improve quality, increase value, all of the things that organizations want to have happen.

Janet Gregory: And one conversation, it can take away so many misunderstandings.

[00:13:30] The Importance of Sociotechnical

Henry Suryawirawan: I think intuitively when people understand that software development is a sociotechnical kind of systems involved, right? They actually understand that they should probably prioritize social aspect more. You know, all these feedback loops, communication, collaboration, psychological safety, and things like that. But actually, many people do not have this same understanding about sociotechnical. And that’s why probably intuitively for them is to actually focus more on the technical practices first. So things like, you know, CI/CD or automated testing and things like that. So I think I like in the book that you actually sequence these technical practices in the order of importance, right? And actually just Selena mentioned, you start with social and then some technical aspects at the end. So maybe a little bit here, like explain why do you think this order of importance is actually very important?

Janet Gregory: Well, so one of the things that I’d be right just said right off the bat, because this is a question I get really often, is we put defect management at the very bottom, right? Because it’s really the lowest. It really is the lowest priority, because if you get all the other things right, all of a sudden you don’t have to worry about defect management. It becomes a non-issue. We debated about putting it in at all! But we put it in priority as if you have this, it makes the next thing easier, right? Because if you don’t have those social aspects, if you don’t have a quality culture, if you don’t have good feedback loops, if you don’t have a good way to learn and improve, without those three social aspects, it doesn’t matter how good you are at your technical aspects, your quality is going to suffer. The quality of the product, the quality of the process. So that’s kind of why we put those ones first, for sure. Do you want to take the other one, Selena?

Selena Delesie: I think you kind of hit it on the head, Janet. Like. to your point, you know, once you get the next things in place, the next things go. Like if your development approach is in good shape, then quality and test ownership can more easily follow, right? And everything just kind of follows through one after another with more ease if we’re getting them in order one at a time. That said, there’s no requirement for any team to work through improvements in their team in that order. They might want to focus on the areas that they’re struggling the most in also, right? But if you’re really starting fresh and you want to do a complete overhaul, like, by all means, work through the priority that, you know, we felt was more, more relevant and important.

Janet Gregory: Yeah. I was just going say that a team may take, for example, development approach and just do an assessment on that to start with.

[00:16:11] QPAM is Not a Maturity Model

Henry Suryawirawan: Yeah. I like the way that you explain, like if you focus on the first few practices first, right? The remaining practices will become easier to implement. I think this, again, intuitively makes sense if you understand this sociotechnical aspect. But for people that also struggle to actually thinking about how to improve this quality assessment, I think this model is kind of like good, right? But I have one question, because typically if we find this kind of model somewhere and you have different stages, they will call this maturity model. Why in your book you specifically mentioned this is not actually a maturity model? So maybe explain a little bit on this.

Selena Delesie: Neither of us are a big fan of maturity models. That’s probably the starting points, because that indicates that if you’re doing things right, you have to be at a certain state. Like you have to go to the highest level. Like you’re not doing a good job otherwise. And we don’t think that’s true. We think that each organization and, you know, a team is unique and the environment that they’re in and the desires that they want for themselves should be respected, right? So for one team, like hitting, moving from beginning to unifying to practicing might be enough in terms of their testing breadth and development approach, for example. And they say, you know what, we’re really happy here and we don’t need to go into the innovation stage, it doesn’t make sense for our product. It doesn’t make sense for our people. And our customers don’t care enough, you know, like it could be something like that, right?

So really, it’s about respecting the uniqueness of each environment and the people in the product, and that they’re able to make those decisions for themselves. The key is being aware of what possibilities might be there for them, and really making a conscious choice rather than just blindly landing somewhere and saying, we’re the best because we’re here, but we don’t know where here is, and we don’t know, you know, what the other options are, right? So we want teams to, organizations ideally, to make conscious decisions about what’s best for them rather than blindly landing somewhere.

Henry Suryawirawan: Yeah, thanks for explaining that. I like the way that you phrase there are options that you can aspire to become, right? And then you consciously make decision rather than maturity model typically is kind of like released by, I don’t know, like consulting or some industry best practices. And management, once they see it, they just say, okay, we want to be the most mature organization ever, right, by following this maturity model.

Selena Delesie: Sorry, can I just add to that? It was like you hit on the head, like it’s not a best practices, right? It’s options available. And when people are ascribing to a maturity model, people will start to game the system so that they believe that they’re high performing or world class when they’re actually not. Because they need to look good to management, right? So moving against that is important and so that people can be honest with themselves and they feel safe to be honest with where they’re at and where do they want to land and it’s okay. It’s okay that they don’t want to be an innovation. It kind of goes in part with the psychological safety aspect of culture, right? And if we can promote a model that espouses that, I think, that’s healthier for everybody too.

Henry Suryawirawan: Yeah. I think the spirit is the most important thing, right? So I find all this kind of assessment typically is like what you said, right? People will just game it because once you make it like a KPI, right? It was just find the metrics, right? What are the metrics that are being assessed and try to game that, you know. And try to actually make people feel that they are actually improving. So I think the spirit is very important and we should not kind of like blame, right? So you mentioned about psychological safety, like not blame the people for the bad practices that they’re doing, simply because if it works for their context, like, why not?

[00:19:52] 4 Dimensions

Henry Suryawirawan: So I think you define the four different dimensions you mentioned, right? It’s not like maturity models again. So there are four different dimensions and it’s kind of like the different terms that you use, maybe a little bit explaining here as well. What are the four different dimensions?

Janet Gregory: Perfect. So we landed on the idea of beginning, because we’ve tossed around a lot of terms. We asked them a lot. We decided not everybody knows where they are. So in the beginning, it’s kind of where are you starting? And usually that will happen and it still amazes me how many teams are still moving to agile that aren’t. Maybe they’ve come from waterfall, maybe they’ve come from some chaos or something else, who knows. But the beginning, where are you starting? You know that you have issues, you want to get better. So we just said, okay, we’re beginning.

And then the unifying is the idea that the team is starting to adapt to agile practices. Whether you call it agile or not, I really don’t care. But they’re starting to think about collaboration more. They’re trying to get those communication lines better. So we talk about unifying. They’re doing the practices. They’re trying to understand.

And then practicing. Practicing is really, all right, we understand. We’re delivering our products on a regular basis, whatever regular basis means. We’re not having that many defects and we’ve got a continuous integration and delivery cycle going. We’re practicing, we’re doing well.

And then innovating is those teams that, and I’ve been on a few and they’re so much fun. But there’s a lot of stress, too, because you’re always pushing the edge and and you’re innovating. You work together as that really great team and you’re just constantly looking for how to better yourself, right? And so that’s innovating. Did I miss anything, Selena?

Selena Delesie: Oh no, these are great. And I would just add, you know, Janet said you could be agile or we don’t care what you call it. I mean, we’ve titled the book Assessing Agile Quality Practices with QPAM, but it doesn’t have to be agile. I mean, the particular practices embodied are just healthy practices for any software technical organization, right? Irregardless of agile, whatever comes after it, it doesn’t really matter. Like these are just healthy practices for any software organization. If we had them in place when I started my career a long long time ago, we didn’t work in agile then, but boy, oh boy, we would have been in better shape if we had been applying this stuff, right? I think the same could be said now or 15 years from now.

Henry Suryawirawan: Thanks for explaining all these different dimensions. I think it kind of like makes sense, right? People always begins when they learn about new practice, right? So you have the beginning, you have then the unifying, right, trying to gather the thoughts, adapting to the practices and understanding, actually, the gist of the practices. And then you move into practices. Maybe when those practices become a habit, right, you kind of like do all the time. And then innovating is always pushing the boundary, continuously improve from what you understand. I think this is probably the Su Ha Ri thing, right? So in agile you have this also loop, right? So yeah, I think I like this dimensions.

[00:23:09] Feedback Loops

Henry Suryawirawan: So maybe if we can, right, let’s just touch on a little bit on few of the practices to give people flavor how they actually use this quality aspects and dimensions. So maybe let’s start with the most important one for sure, right? The feedback loops. We kind of like touched on a little bit here and there about it. So feedback loops is also one thing that kind of like brought up quite a lot recently about developer experience, about also the flow things, right? And also continuous delivery and all that all gathers around feedback loops. What is the most important thing here? How can people assess their feedback loops against all these four dimensions?

Janet Gregory: All right, so we’ve divided it up. It was really tough to try to figure out practices around feedback loops. So we talked about within the delivery team themselves, things like how do programmers and testers talk to each other or anybody else on the team? So in a beginning, they probably have a, you know, throw it over the wall kind of thing. They don’t talk to each other. They report lots of defects, things like that. How are they actually talking to each other? Then we have the idea of between customers, stakeholders, and the delivery team. How do you interact with your customers? Do you even talk to them? Do you listen to what they have to say? Like if you were giving a demo and I watch teams, I’ve been on teams, where you give a demo, the stakeholders have lots of good things and nobody writes anything down. They don’t pay attention. And then the third one that we talked about is between leadership and the delivery team. So in a beginning kind of thing, it’s probably a hierarchical, one dimensional. The leadership team says, hey, thou shalt do this. And everybody else goes running and does that. Changes priorities a million times because somebody tells them to.

And so, when you’re looking at those different aspects and those particular practices, you can kind of see that as, say, a practicing team will have multidirectional kind of feedback loops with their leadership team. They are not afraid, because they have psychological safety. They’re not afraid to say, hey, you know what? That won’t work. Can we try this? And improve those kind of things. Whereas, at the beginning, you’re really just doing what somebody tells you to do. So that’s how we’re assessing those feedback loops on those three dimensions, those three practices.

[00:25:45] Explaining Feedback Loops

Henry Suryawirawan: Yeah I think feedback loops just like for any kind of leaders or management, right, maybe again like they are not aware of about this concept. They think it’s fluffy right, what do you mean by feedback loops, right? So I think it’s not easily quantifiable. So maybe for management or people when you want to explain you should improve your feedback loop. Because like feedback loop, I think it’s very, very evident, you know, in agile practices, DevOps practices, this is probably the one thing that they focus a lot on, right? But for management to actually understand the concept of feedback loops, how would you explain this? So think of like when you do this assessment and you have to explain why feedback loop is the most important thing.

Selena Delesie: Well, it’s all about checking in and getting feedback. Um, Right, so basically, we’re testing ideas, basically, as we’re going. So, you know, as a tester to a programmer, a business analyst to a tester, let’s say, like we’re asking questions of each other frequently day to day about what we’re going on and what’s our idea and what’s our design. And checking those assumptions as we go, right? And then the programmer might be saying, hey, tester, come sit over here. I have some questions about what I’m doing and writing the code. Can you pair with me on that? So we’re constantly checking those things and it’s the same with as we’re communicating with stakeholders or customers or management to the team that we are ensuring that we’re constantly checking in to make sure that we’re on the optimal path to shorten our cycle time and do that with high quality so that our customers get high value.

So in the book, we listed a lot of questions that could be asked for each of the specific areas. And some of them are a little qualitative, maybe a little fuzzier feeling. But how does a tester or team member know what is ready for testing? I mean, like, what is your tool? What is your source? Is it a communication? Is it a sticky note on a wall? Is it something on a task board? Is it something that comes through where the code is stored? Like checking in on how do we know when something’s ready to test? Does the team document decisions that they make, right? So some of them are yes or no. And then where do we do that?

It’s really like, there’s a really good, comprehensive, holistic set of questions that we’ve included to really check in on. And I think that the questions really get into the meat of what it is that you would want to be looking for for any of the particular areas. But what it really comes down to, I think, if we’re talking to senior management, like, what do they care about? Get it done faster, save us money, make our customers happy. Like there’s other things, of course, too, but those are often the predominant forces. And it seems a little counterintuitive that having more conversations and more check-ins is going to get us the fastest throughput with higher value and reduce our costs.

But there’s good data that you could find in your own organization, never mind from other ones that show that that is the case, because if we go too far down a wrong path, then you’re actually increasing time, you’re increasing costs, and, you know, down the road, maybe reducing the value that your customer is going to get, because you didn’t give them what they actually wanted or needed. So those checkpoints are really helpful.

It’s akin to like going on a road trip. And, you know, you have an idea of where you are and where you want to go, but you never check the map. And you don’t check the map anywhere along the route to know that maybe you’re 300 miles off track. Like you want to check in and know that you’re actually on the right path. And maybe that fits for people who drove without using a GPS mapping tool back in the day, like I did once upon a time. But it’s same thing, like…

Janet Gregory: Yeah.

Selena Delesie: Pardon?

Janet Gregory: One of my favorite questions to ask to help them think about what feedback loops is, is how does leadership get the information they need to make decisions? That tells you lots about feedback loops between leadership and the delivery team. That’s what we’re talking about too, right?

Henry Suryawirawan: I love that question. So actually that make you reflect a lot about how do you get the current situation of the team, right? Where the team at? What kind of struggle, challenges that they’re currently facing, right? And I like the way Selena mentioned, right? So it’s not about feedback loop of delivering features fast. It’s actually testing ideas, knowing the direction, right? Knowing that actually you are moving still in the right direction. I think these are all very, very important understanding that we have to have about feedback loops. So I think thanks for sharing that.

[00:30:18] Development Approach

Henry Suryawirawan: So maybe the other quality aspect that I want to touch on is about development approach. So maybe when people talk about development approach, they look at the developers. How are you writing code or maybe things like architecture design? So maybe something more that you touch on in the book, right? Development approach is not just about writing code. So maybe brief us about this.

Janet Gregory: So development approaches. How does the team… and we call it the delivery team. How do they work together? How is the team made up? Is it remote? Is it together? Like, those things will affect how the team works. Doesn’t mean there’s a right and wrong. It just does. So things like quality ownership. Does the team really think that the testers own quality? And when we say delivery team, it’s programmers, testers, analysts, product owners. Who is on your team and, and how do they do that? Thinking about what practices they do. Like if you have a, whatever agile format you’re using will depend on the practices. If you’re doing Kanban versus Scrum versus Extreme Programming versus whatever, so there will be different practices and we don’t specify those specifically.

We just kind of, what’s important to you, trying to understand what that is. Do you have some kind of prioritization on your stories or your features you’re bringing in. How do you handle those sorts of things? Do you know when a story is done? What does that mean to you? How do you work with delivering it to production? How does the team work together? So those sorts of things when we’re looking at a development approach is right from the very beginning, do you have a mechanism in there for asking the questions right through to delivery? How do you deliver to production? What is that?

Henry Suryawirawan: Yeah. So I think it’s really important that the team actually have a mechanism or the practices in place, right? So first of all, it’s like ad hoc, right? Like people just do whatever practice that they think they want to do. But actually have a method, an approach, right? And then like, you also holistically look at not just the code aspect, but also like things like gathering the requirements, right? The ownership about the code and testing part. And also like, how do you release that story or features to the customers, right? And then get the feedback loop, again, back to the team. So I think development approach here is really important because I find so many teams probably, like they just take agile framework, any agile framework, and they think that’s the development approach, right? Take Scrum, so they just say, yeah, we do stand up, retrospectives, and, you know, product backlog grooming, things like that. And they think that’s enough. So I think the point here is that it’s more beyond just the code and the framework but it encompasses so many other aspects as well.

[00:33:03] Defect Management

Henry Suryawirawan: So maybe the other quality aspects that I’d like to touch on is about the defect management. Like interestingly, Janet, in beginning, you mentioned that you actually thought about removing this from the quality aspects. But for people actually to assess quality, normally, again, intuitively, the first thing they will do is, how do we track defects? How do we log defects? How do we know that they are closed? So maybe explain why defect management is put as a lowest order in the priority.

Selena Delesie: Yeah. So with defect management, it’s A, why it’s lowest down, is because if we’re doing really well in the other areas, we’ve improved in a way that the other areas are feeling really strong, the amount of defects that we have should be going down dramatically. So we will have less of a need to track and manage those things. Certainly, there may be some escapes, but we shouldn’t be going intentionally into releasing things with a lot of known issues, ideally, right? So, if we’re doing really well in the other areas, our need for management is reduced greatly.

One of the ways that we do that is when we find things, when we are in the process of developing, which includes the programming and the understanding and the testing, we fix them right away. We don’t consider things to be completed until we’ve addressed the issues that we found, right? In which case, what we have is defects out in the world or things that are being found by people outside of the team, in general. And again, those should be a lot less if we’re doing the other things really well. So from there, we’re looking at issues around how do we report defects? How do we triage them? And how do we report on them? In some organizations, they will choose to use defect management tools, still. One of the things that we recommend that we’ve seen work really well is that defects get prioritized right alongside with new features.

So we’re making decisions and we’re storing those things right alongside with any new features for the products that we have in the team’s backlog. So there’s a clear decision about what’s more valuable. This new feature or fixing what a customer has reported? And that in itself takes away the need to have a separate defect management tool. That’s one option that we’ve both seen work really well. The time that’s invested in the reporting and fixing and triaging, I know early in my career, we spent a lot of time as testers, as programmers, as project managers, as all of these different roles in meetings and in sending emails and in dealing with the stuff, because there was a lot of problems, right?

And at one point, I was able to track in the organization I was in of several thousand people, 40 to 50% of people’s time is spent on these activities, right? So it really, really, really is very helpful to focus on the other quality aspects and doing them well, so that you can get that time back, because just focusing on the defects and managing them and tracking them isn’t going to save you the time and the money that you want to save. You want to focus on fixing the other, quality challenges you have in your process within your organization and that in itself will address that and hopefully reduce it dramatically, which I did see happen in organizations that I worked in.

Janet Gregory: Yeah, I’m only gonna add one thing. I think if you measure the defects, it tells you how bad the quality is. It does not tell you how good it is.

Selena Delesie: Excellent closing comments.

Henry Suryawirawan: Yep. Wow. I think that’s, again, pretty deep, right? So you only know how bad you’re doing, but actually not how good you’re doing. And I think in your book, you mentioned that the goal is actually for defect prevention, right, not actually detect detection. I think many people try to detect a lot of defects, right? Manage it, report that. I think when you mentioned about reporting defects, I kind of like laughed, because in the past I also used to experience those kind of things. And not just the report, but also the fighting in-between, right? When the testers report something and developers don’t agree, you kind of like fight instead of prioritizing.

So I think all this really important because in many teams culture, right, still defect management, you know, the JIRA or whatever tools that they use is probably the center of their quality, so to speak, right?

[00:37:19] Understanding the Problem

Henry Suryawirawan: And I think very importantly in your book, you also mentioned that we should actually spend more time in understanding the kind of problems, the features that we’re doing and the kind of testing that we want to build from the very beginning, right? Not just something that you throw over the wall or you think about it at the end. So maybe a little bit on this part, how can people start thinking about understanding the problem first and, you know, like building the test cases in the early phase rather than the last?

Janet Gregory: Right, so I’m going to tell a story here. And people don’t think about this necessarily as testing. But I do because, yeah, it’s my life. But I was working with a team one time, and they were all ready to start on a feature. They were going to have their kickoff meeting and just start going with this new feature. Product owner had done a bunch of work, they had stories ready to go, and I had been trying to coach them a little bit on maybe let’s start a little earlier thinking about some of these things. So I said, before we do that, can we just take time, do it with a half an hour, and let’s build a mind map. The product owner understands this feature really, really well. The team hasn’t seen it so much, so they’ll be learning about it but asking questions, trying to understand. So can we just have a quick half hour meeting and build a mind map on what this feature is?

And so, you know, the product owner was a little not sure, but he agreed and the team said, okay, we’re there willing to try something new. So we had this half and we spent half an hour building a mind map. It was remote, we did it online. Before we finished that half hour, there were so many questions that had come up from the team. They were bringing up risks, asking questions that had not been thought about before, that they actually took that feature off the table and said, we’re not ready. So that half hour prevented probably weeks and weeks and weeks of blocking stories, trying to understand this. They just took it off the table and went back and started asking some of those questions. That, to me, is testing early. We’re testing our ideas. We’re testing our understanding. We’re exposing risks. It’s just not testing the software.

Henry Suryawirawan: I think that’s a really powerful story that we can learn from, right? So I think this also comes back to the holistic testing approach. I think we covered that in the previous episode as well, right? So testing is not just testing the code, testing the product, the feature, but also testing the risk, the ideas, right? And make it early, as early as possible. I think it’s a very very important. So I really love the story, right? So I think it’s quite typical in many software teams, right? Some product owners or stakeholders will just say, okay build me this feature, like in a few sentences. Maybe better if they have “as a something, I want something, so that something”, right? But it’s always not enough. They seem to miss the details, the edge cases, and maybe the variance of how the user journey will look like by using those features. So I think thanks for raising that.

[00:40:26] Conducting a Quality Assessment

Henry Suryawirawan: So knowing all these quality aspects, then the next important thing is actually to conduct the assessment itself, right? For teams who probably don’t have expertise, they cannot hire coaches like yourself. How would you advise people? Because I know that you are also writing the second book, right? To actually help people facilitate this kind of assessment.

Selena Delesie: Yeah. So in the first book, the Assessing Agile Quality Practices, we do have a chapter devoted to how to conduct a quality assessment. It’s a brief synopsis of what we would recommend. And our second book, A Guide for Facilitating Quality Assessments, goes into super detail. Everything that we would recommend, whether you are an external consultant, assessor, facilitator, or internal, or you are a team member, and there’s actually a whole chapter devoted to I’m a team member. How do we want to do this as a team? So the steps are gather information. How do we prepare for the assessments? Then we actually conduct the assessment. We gather everything that we need to do. And then how do we put all the information together in a way that is meaningful and assess all of the information that we pulled together. And then how do we want to report in the results for the team’s benefit, for the organization’s benefits, and then any follow up activities that you might want to do.

So there’s a lot involved with each of those steps, whether it’s how do you identify who to include in the assessment? Do you have examples of like stories or tests or code maybe that you want to look at? Do they do things like mind mapping? If so, let’s see examples of that. I don’t know. There are examples. I used to work for a company who we did use that sort of thing. And I know Janet has too, but that’s not true everywhere, right? But if they have examples, you want to see that. And setting up all of the meeting invitations, getting the right people in the room, whether you’re in a room physically or doing it virtually. And there’s also things like doing one-on-one discussions with people, like a selection, a subset of people. Or you’re doing group interviews with, say, maybe all of the testers or all of the managers who are involved or whomever it makes sense to have those discussions with. But there’s a lot of sources that we’re pulling together.

One of the really bigger activities that we would do in terms of conducting is what is called a process retrospective. So understanding from the very beginning of a project that we work on all the way through to delivery, what are all of the steps and stages and activities that take place. Understand like how it all fits together with all of the people involved in the room together. And oftentimes you will see light bulbs or questions or confusion pieces go off around, I didn’t know that was happening, or what do you mean that’s what you do next? Like, it’s a really great way for the whole team and the project team or the delivery team to get on the same page about what is actually going on, because they don’t often all know that, right? Yeah, and then pulling it together into reports. We actually provide a tool that you can do that with for those who’ve purchased the book. They can get the link. It’s a free downloadable spreadsheet that you can use to help you to do your assessment. And analyze your results.

So, yeah, for us, it’s like, yes, we can do assessments. I know Janet’s not anymore. She’s in her retirement phase. But like myself and her other co-author, Lisa Crispin, and like anyone who’s picked up the book, like frankly you can pick it up and run with it and be able to facilitate an assessment. But you’re more than welcome to involve people, who have done these before too to learn from others who’ve done this a few times.

Henry Suryawirawan: I like the way that you, when you describe a process retrospectives, right? So sounds like a value stream mapping, by the way, right? If people

Janet Gregory: It was kind of. It’s a little bit…

Selena Delesie: It’s similar. Similar, yeah.

Henry Suryawirawan: So it’s always a very revealing session where people start to understand, oh, what do you mean that we have this? Or what do you mean we are doing those kind of stuff that are unnecessary, right? And it’s not just for the team, right? The leaders also they sometimes don’t actually have a good understanding what the team is actually doing and what are the process involved. Or maybe those kind of habitual things that are probably legacy or bureaucracy. So actually they may not even need to do that anymore, right? You can take the decision to actually remove that from the process.

[00:44:49] Insights from Past Assessments

Henry Suryawirawan: Maybe any kind of interesting learning. I know that probably you have done these assessments a few times with your customers. Any kind of revealing insights or revelation from your customers after they’re doing this kind of assessment? Maybe if you can share one or two.

Janet Gregory: I think one of the biggest things that I’ve seen most often, actually, is the programmers saying, I didn’t know that you did all of those things. Because testing is always, it seems to be one of those hidden things. It’s just magic. And so when you start putting up all of the little things that is being done, you will get a lot of that. “I just didn’t know.” We can help you with that. That’s what you want to hear, right? Yeah.

Selena Delesie: I mean, I’ve heard that one too. Absolutely. I mean, the biggest aha really is just having it all there together in one visual picture. It is really just a big aha for everyone in the room about what it really takes to do the work that we’ve been asked to do. That sounds like it’s just, oh, just put this change in really quickly. Da da da! And it’s all beautiful, right? It’s not that simple. It’s not that easy. And when we know all of the steps that are involved, yeah, there are ways that we can find a way to accelerate in healthy ways some of those things by automating the right things at the right time. But we have to be mindful about how we build quality into those steps too, right?

So it’s not just we snap our fingers and it’s out the door, right? Even with things like bringing AI in into organizations now, like there are things that can be beneficial and helpful, which I mean, we didn’t touch on in this book, but it’s the same. It’s automation, right? How do we bring automation in to support us and help us? And there are considerations that we need to make to ensure that we’re making good choices. So regardless of how we look at the process and see how big it is and how involved it is and how complicated and all of these lines of communication and feedback loops that need to happen all over, ensuring that we are front loading and addressing quality rather than leaving it to the end is really what we want to see people doing, so that they are able to be as effective as possible.

And in my experience, an activity like this helps people to start to make that connection and understand how to get there. Because now that they’ve looked at it, they’ve had an assessment, they’ve gotten their results back, and they’re like, oh, okay, we were really struggling in like development approach, testing breadth, and feedback loops. Let’s start with feedback loops first, and then maybe the other ones will kind of domino effect down, right? And then they can make conscious choices about that. But like having those results and really understanding where they are and why they’re having the problems that they’re having. Yeah, that’s the big aha that I see with teams and organizations.

And I would say with management, in particular, because they often don’t understand everything that’s involved. And this gives them something tangible to hold on to, to understand how they can get from this unknown, like chaotic, I don’t know why we are here, but we’re still consistently not doing a good job into this is why we are here. And these are the steps that we can take to get to where we want to go. So there’s a lot of ideas. But all of them, there’s a few key things there, I hope that stood out.

Henry Suryawirawan: Yeah, I think it would be really cool if every team has this kind of assessment. You know, they put together all these kind of quality aspects. Where they are at? I think, again, a lot of teams probably don’t even understand where they’re at, right? And putting this together, putting it as a map, right? And I think the most important thing also, like the roadmap where, for example, if you’re in the beginning stage, you know there’s a roadmap that you can aspire to, to become the innovating dimension, right? So I think having this kind of roadmap that you put in place for a continuous improvement process within the team will be really, really something that is cool. I tried to build that. I know it’s probably failed more than a successful one. But I think, yeah, it’s really hard to come up with this kind of assessment model. And I encourage people to check out Janet’s and Selena’s book, right? And maybe try to use that to assess where you are.

[00:49:04] 3 Tech Lead Wisdom

Henry Suryawirawan: So we reached the end of our conversation, but before I let you go, I have one last question. Janet, maybe you still remember last time I asked this. I call this three technical leadership wisdom. So think of it like the last advice that you want to give to the listeners. Maybe you can share the version of three technical leadership wisdom from any of you?

Janet Gregory: Okay, I’ll give one because it comes right out of what we just said, what Selena just said. Visualize. Visualize whatever, whenever you possibly can. Because, for example, if you can see a process, then you can discuss it and decide where you want to take it from now. So I’m a big fan of visualizing it, trying to draw a picture of it, whether it’s a mind map, a flow diagram, whatever makes sense.

Selena Delesie: There’s so many things that we could add. I would say working with reality, I think, is really important. And the tools, the model and some of the tools that we talked about around assessments, I’m including visual, like, visualizing, but, where you are. Like if you just got dropped out of a plane with a blindfold on you, and you don’t know where you are in the world, you want to figure out where you are before you make plans about what to do next, right? And it’s the same within the context of your organization and the teams and your products. Like, taking some time, and it feels like a slowdown and you’re too busy to do that, but taking that time to know where you are and know your options for getting to where you want to go is really important. And I see a lot of teams and organizations struggle to do that. And that’s why consultants and coaches have jobs. Because we help facilitate that process to understand where you are and where do you want to go next. So yeah, that’s really important to do.

Janet Gregory: I’m going to add a third one, which I’m sure Selena will agree with. Don’t be afraid to question, especially, if something is making you uncomfortable or is pushing your own ethics, your own morals. Sometimes it’s, it’s just something and you’re scared to ask that question, because you think it’s a dumb question. But often, that’s the thing that can make a team just stop and say, we never thought of that. So break out of your comfort zone and ask the question.

Selena Delesie: One reframe of that is like have courage. But it’s not even that. It’s, you know, having the willingness to sit in the uncomfortable space that you’re in and sitting in uncertainty. And with the questions that you’re asking, don’t ask why questions. Go a little deeper than that.

Henry Suryawirawan: Yeah. It kind of like also shows vulnerability, right? I think Brene Brown has this concept, right? So the courage is actually the courage to actually be vulnerable, right? Show vulnerability and ask these kind of questions, right?

I think it’s really important in a team where you have a lot of psychological safety. This will be probably a key thing where you can start becoming a high performing one. So really beautiful tech lead wisdom.

So for people who want to check out the book or they want to learn more, any kind of resources that you have available online?

Selena Delesie: Yeah, so we have um, both books that are available for purchase on LeanPub, so you can look up our names or QPAM. Both books will be coming up. So Assessing Agile Quality Practices with QPAM and A Guide for Facilitating Quality Assessments. Both books are also available for purchase on Amazon worldwide as a physical copy, so if you want a physical one, that’s where you can go buy that. Otherwise, website is forthcoming. That is our focus for the next few months. And yeah, otherwise you can find more about me on LinkedIn at Selena Delesie. And Janet?

Janet Gregory: You can find me pretty much anywhere. My website is janetgregory. ca, .ca. So yeah. From there you can find me anywhere, but if you google me, I usually come up pretty quick, so.

Henry Suryawirawan: Yeah. I can attest to that. So your name and Lisa Crispin are always like the thought leaders in the testing space. So I’ll make sure to put all those resources you mentioned on the show notes, right? Thank you again so much for this conversation. I hope people learn a lot of things about quality aspects and the dimensions and people start to assess where they’re at so that you can improve from where you’re at to something that you can aspire to become in the future. So thanks again for your time.

Janet Gregory: Thank you very much for having us. Yes.

Selena Delesie: Yes.

– End –