#133 - Software Requirements Essentials - Karl Wiegers

[00:01:06] Episode Introduction

Henry Suryawirawan: Hello again to all of you, my friends and my listeners. Welcome to the Tech Lead Journal podcast, the podcast where you can learn about technical leadership and excellence from my conversation with great thought leaders in the tech industry. If you haven’t, please subscribe and follow the show on your podcast app and social media on LinkedIn, Twitter, and Instagram. And to appreciate and support my work, subscribe as a patron at techleadjournal.dev/patron or buy me a coffee at techleadjournal.dev/tip.

My guest for today’s episode is Karl Wiegers. Karl is the coauthor of “Software Requirements Essentials” and has previously appeared in our episode 103. In this episode, we discussed 6 essential practices for software requirements out of the 20 core practices specified in his book, such as understanding the problem before converging on a solution, understanding what users need to do with the solution, prioritizing requirements, and a few more. Karl also explained the importance of having a clear and effective communication in developing software requirements, his view on doing software requirements for Agile teams, and the importance of having good software requirements for becoming an effective software development team and for avoiding unnecessary rework.

It’s really great to have Karl back on the show for the second time. And I always enjoy and learn a lot from my conversation with him. And I hope you also enjoy this episode as much as I am producing it. Please share this episode with your colleagues, friends, and communities, and also leave a five star rating and review on Apple Podcasts and Spotify. And I hope more people can benefit from listening to this episode. Before we go to my conversation with Karl, let’s hear a few words from our sponsors.

[00:03:21] Introduction

Henry Suryawirawan: Hey guys, welcome back to Tech Lead Journal podcast. Today, we have a repeat guest, someone that I really loved the conversation last time in episode 103. He’s Karl Wiegers, so if you still remember, we talked about software pearls. Last time, he published the book. And this time he came back with another book. If you still remember, he has written 13 books back then, right? So he decided to be productive and write one more to become 14 books now. And his new book is titled “Software Requirements Essentials: Core Practices for Successful Business Analysis”. Karl, I’m really excited to have this chance to talk to you again, and hopefully, we can talk a lot of things about software requirements today. Welcome to the show.

Karl Wiegers: Well, thank you, Henry. I appreciate the chance to be back with you. Like you said, I enjoyed our conversation before, and I’m sure we’ll have a good time today as well.

[00:04:08] What Karl is Up to Lately

Henry Suryawirawan: So Karl, it’s been quite a while since our last speak, right. Is there anything that you do recently that you want to talk about besides the book?

Karl Wiegers: Well, when I’m not writing books or articles, I do like to record songs. That’s sort of my main hobby. My wife told me, probably about 17 years ago now, she said, “Karl, you need a new hobby.” Well, I’ve played guitar for many, many years since I was 12, and I’m a lot more than 12 now. But I started recording songs just for fun.

So I got some software and a computer and I enjoy trying to reproduce covers of original songs that other people recorded, trying to mimic what they did as well as I could. Plus, I like to write my own music and record that too. So it’s kind of fun to do all the guitar parts and the bass and the vocals and keyboards and stuff and program the drums.

So I’ve done a number of songs since last we spoke, and they’re all at my website, karlwiegers.com and they’re just for fun. I’m certainly no professional, but it’s a nice creative outlet and always a good learning opportunity.

Henry Suryawirawan: Yeah, I still remember I could find a number of songs on your website. I think you also listed, I think about 50 songs there, right? So I think not just writing books, you also write songs. So that’s really great.

[00:05:20] Writing Software Requirements Essentials

Henry Suryawirawan: So Karl, let’s go to your book, right? So you wrote this Software Requirements Essentials. If I’m not wrong, you actually wrote the thicker version of it. It’s called Software Requirements, which has gone through multiple editions as well. Maybe if you can give us a background why you decided to write a thinner version of that book instead.

Karl Wiegers: Yeah, it was kind of funny. I hadn’t planned to write this book, but nearly a year ago I wrote a short article titled “The Six Most Important Requirements Practices”. Sometimes, I just feel like writing something or get an idea, and so I wrote this little article. It was only maybe 12 or 1300 words, and I posted that article on my medium.com account. I have nearly a hundred articles up there on many topics, and I was just completely shocked. The response to that short article was just astonishing. It’s been read more than 18,000 times. The posts that I created on LinkedIn to announce the article have been viewed about a hundred thousand times with, you know, about a thousand likes and many reposts. And I was just completely astonished at the positive reaction to just that little article on the six most important requirements practices.

But that made me start thinking. Maybe there’s a market for a short book on requirements that just hits the highlights. Because like you said, I wrote this book called Software Requirements in three editions. The third edition came out 10 years ago. And that was a big book and there were a lot of long comprehensive books on software requirements and business analysis. And those books described dozens and dozens of practices. All these things a business analyst or a product owner or a product manager ought to do. And those are all useful in various situations, but busy people don’t always have time to read these long books.

So I had this idea of kind of a paradigm shift, writing a short book that just skims across the top. It describes the 20 most important requirements practices that I think apply to virtually all projects and teams. So that was my idea. And, you know, it was nice because the book just came out a few weeks ago. And when I held my copy of the book in my hands and looked through it, I realized that was exactly the book that I was envisioning last summer when I first got this idea. So it’s nice when it works out that way and you say, yep, yep, that’s what I was trying to do. So I’m very pleased with how that came out.

And I chose to work with a co-author. I did that on the second edition or the third edition of the Software Requirements book. But I enlisted the assistance of a co-author, Candase Hokanson. And partly cause she had a lot of experience with product ownership and business analysis on Agile projects, which I don’t have. And Candace brought a lot to the table. She brought like any other co-author, different experiences, new true life stories to share beyond my own experiences and what I’ve learned from other consulting clients. She has a lot of hands-on current project experience that I don’t have personally these days. So she was helpful in making sure that the practices we selected as being in the top 20 apply to virtually all software projects regardless of the kind of product you’re building and whether you’re following an agile or traditional development approach. So that was what we set out to do and I think we did it.

Henry Suryawirawan: Right. Just maybe to give some illustration for people who are not famliar with your Software Requirements book, if you compare the pages, do you know how many pages difference is it?

Karl Wiegers: Yeah, it’s a pretty big difference. The third edition of the Software Requirements book, which I also co-authored with Joy Beatty, is a seriously big book. It’s more than 600 pages of content. But it sort of had to be because it comprehensively covers the entire domain of requirements development and management. And the fact is, there’s a lot of information, there’s a lot that people need to know about these things. There’s a lot of things people need to do, and requirements really affect everyone on the project, all the stakeholders on the project. Requirements is where their interests all intersect. So, that’s a lot to read and many people have read it, but what about something shorter?

Well, in contrast, the Software Requirements Essentials book is only about 150 pages long. It’s about one quarter as large. So obviously there’s a lot more information in the big book, and we didn’t try to duplicate all of that. But we don’t have as many topics. We don’t go into as much detail. We don’t have as many stories and appendices and all the other things you find in a comprehensive book.

But the Software Requirements Essentials book is really an excellent resource for people who want to quickly learn about these most important core practices that can add value to their projects and help keep them out of trouble. And then you can go back to the big Software Requirements book or other resources for details when you need them.

Henry Suryawirawan: Right. I had a read for the new book as well, so I think it’s pretty short, right? Like you said, I think someone who likes to read can easily finish it in a day or two. So I think it covers a lot of essentials, just like the title, right? The Essentials of Software Requirements, which we are going to cover now.

[00:10:19] Software Requirement

Henry Suryawirawan: So for people who are maybe a little bit confused about what is software requirements or they had been burned before about bad software requirements, so maybe if you can elaborate, what is a software requirement?

Karl Wiegers: Well, that’s an interesting question, cause it seems like there should be a really simple, short answer, but there’s not. And I think the first thing people have to understand when they start talking about requirements on any project is that people have different ideas of what that word means. And so we immediately have an uphill battle because people, you know, one person says, “Well, let’s talk about requirements.” And different people in that conversation have different ideas of what kind of knowledge, what kind of information that includes. And so I think people have to learn that there are different kinds of requirements information. We have to put some adjectives in front of the word requirements.

So we have to think about, for example, business requirements. And to me that’s talking about, really, answering the question of why are we working on this project? What are our objectives that led us to decide that this was an important project to start with. And then there are user requirements which talk about the things people will be able to do with the product or the solution, whatever it is you’re building.

But then there’s other details that are the kinds of things that developers need to know about. They need to know about the functional requirements. The basic ways the system is supposed to behave under certain circumstances. And there’s also non-functional requirements. A lot of people don’t like that term, non-functional, because it tells you what something isn’t rather than what it is. But when people say non-functional requirements, they’re usually thinking of what are more commonly called quality attributes. And these are things like the -ilities, you know, usability, portability, and so forth that describe not what the product does, but how well it does these things. In fact, we’ve all used products that did just what they were supposed to do, but we hated them cause maybe they crashed too often or maybe they were too slow or they took too many steps to get something done. So those are examples of products that meet their functional requirements, but not our quality expectations.

And then we have constraints to worry about and external interface requirements. So there’s all these kinds of requirements information that we have to deal with. But basically what we’re trying to do is, with any requirements conversation, is understand a need and then conceive of ways to describe to construction people, like developers, what they have to build in a solution to satisfy those needs.

Henry Suryawirawan: So I think what you said speaks true. Like there are many different types of requirements. Probably also some people, they didn’t have a chance to have all these comprehensive requirements. And that’s why I think requirements sometimes in any software team becomes a challenge, right? So the software, first of all, why it was being built, right? First, it must come from requirements, but sometimes requirements is vague. Sometimes requirements just few lines of statement of objectives, right? And I think that’s where the challenge is.

[00:13:20] Clear & Effective Communication

Henry Suryawirawan: And I think, I still remember in our previous episode you mentioned that the goal of a requirement is actually to have a clear and effective communication. Maybe if you can touch on a little bit about this again, right? Why you think requirements is very, very important for clear and effective communication.

Karl Wiegers: Yeah, that’s exactly right, Henry. In fact, requirements are not really about technical things. Requirements are a communication problem, not a technical problem. And that’s, I think, why it’s hard. Because rather than just saying, okay, we understand what to build, let’s go figure out the right language to use and the right code to write and what platform to run on, all that kind of stuff. It’s not like that. It’s a matter of human communication and understanding a problem and exploration.

If you go and ask someone during a conversation, what are your requirements or what do you want, those are terrible questions to ask, because nobody really knows how to answer those questions. And so we need better questions to try to help explore what people have in mind and why they have it in mind. And then we can use that information to conceive solutions, hopefully multiple possible solutions, so we can evaluate them and choose the most appropriate one. But that’s why it’s hard.

And one of the results of that, or one of the consequences is that, it has to be done iteratively. Ideally, sure, we could just go ask somebody, “What do you want?” They’ll tell us, we’ll go off and build it. And that just simply doesn’t work. We know that doesn’t work. And so, this process of requirements development, where our goal is exactly as you said, clear and effective communication. It’s not following a set of rules. It’s not conforming to some standard. It’s effective communication. And that has to necessarily be done in an incremental and iterative fashion.

And that’s frustrating for people. Because you go and ask someone for information, you get some, and then as the business analyst (and no matter what your job title is, if you’re doing this kind of work, you’re functioning as a business analyst; that’s a role, not a job title) – so you go think about it and try to understand it, and then you realize, oh, there’s something you thought you understood during the conversation, but you really don’t. Or you have questions. Or you realize it’s the opposite of what somebody else told you the next day. So you have to go back and revisit things. You have to go down layer by layer to get more information and resolve conflicts and all that sort of stuff. And so that’s an iterative process that can be frustrating for people who might say, “Well, wait, I already told you this. Just go away and call me when you’re done.” And that’s just not realistic. So this is why it’s a hard problem. And I think people just have to acknowledge the reality of the incremental and iterative approach and the fact that it’s relying on imperfect human thinking and communication. There’s no way around that.

Henry Suryawirawan: Right. Not to mention you also have to write it, right? Not just communicating, asking people the questions, but you also have to write it so that the developers or whoever in the downstream is able to understand the requirements itself and be able to build and test it, right?

[00:16:19] Importance of Requirement Details

Henry Suryawirawan: So I think, when you mention about asking people what they want, what kind of requirements you want, I think some executives or some leaders who have the ideas, sometimes are not very interested in detailing the requirements. I don’t know whether it’s a problem of time or it’s something that even they don’t have clear idea, just like any great ideas, right? It’s just like a spark of an idea, but you don’t have the lower level details. What would be your advice for people who seem to have these all great ideas, but they’re not really interested in detailing further into requirements so that maybe the business analyst job becomes much easier. Any tips or advice from your experience here?

Karl Wiegers: Well, yeah, I think it’s great if we could rely on telepathy, just exchanging information mentally. Or if we could rely on clairvoyance where someone just automatically can see the future and knows what people need and can fill in all the blanks. But the fact is those are not very good requirements elicitation techniques. I don’t recommend clairvoyance and telepathy, although on some projects that does seem to be the technical foundation. It doesn’t work very well.

And so I think it’s fine to have people who have visions of a product, a solution that’s going to hopefully meet some need. And I think it’s very helpful if we can first understand what problem this is solving. In fact, someone, a senior person or a visionary might come and say, “Hey, build me this.” In other words, they present you not with a need, but with a solution they have in mind. And I think it’s dangerous to either assume that the presented problem is necessarily correct or that the presented solution is necessarily appropriate.

So, the question I think of in that situation is, well, if that solution was the answer, what was the question? Let’s back up a step. So I’m a big fan of doing some root cause analysis to make sure we understand the problem first and then make sure we can address the correct problem. But, the fact is we do have to get down to detail. Somebody has to answer those questions. Somebody has to bridge the gap between, okay, I have an idea for a product, down to, well, what color should this text be? You know, in between there, there’s a whole lot of conversation and thinking that has to take place.

And frankly, I think maybe that’s one reason why sometimes people don’t want to do that, because thinking is hard. Ideating is not as hard in some ways, but thinking of all the details, the implications, the connections, the elaboration – that’s hard. And it’s frankly not all that much fun. Compared to just coming up with a cool idea for a new app, you know.

So I think we do need to get to that point where we understand all those details. Cause developers are going to build one thing. They’re just building one thing. And I’ve heard it said once that the requirements may be fuzzy, may be general, may be incomplete, but the product will be very specific. It will be one thing. So if we want to make sure we get the right thing, at least eventually, then we’re gonna need the information to be able to do that. And this is true of any project, of any product, of any culture, any development approach. Those things are true. And that’s something we tried hard to do with this book, is make sure that the 20 practices that we selected, the techniques we describe around each practice, really could apply to almost any situation. They’re very, very universal.

But the other thing you got to a little bit earlier, Henry, that I want to come back to is the idea that we need to write some of this down, because not only is telepathy not very reliable, but neither are human memories. I don’t know if you’ve ever been in a situation where maybe you were in a meeting and some people walked out of the meeting and they later on discovered they had different recollections of decisions that were made, and what came out of that, and what they’re supposed to do next.

Whenever we have those kinds of differences in our memories, we’ve got a problem. So, I think an important thing for people to remember is that the cost of recording knowledge, that is writing it down or otherwise storing it, the cost of recording knowledge is small compared to the cost of acquiring the knowledge. So I don’t frankly have a lot of patience with people who say, well, we don’t have time to write this down, because they’re setting themselves up for problems in the future. And basically they’re saying, well, we don’t have time to write this down, but we’ll have time to figure it out again later. That’s not a very good way to think about it.

Henry Suryawirawan: Yeah, we kind of covered this as well in the previous episode, right? So I think it’s really important for people. The cost of recording information is actually small. Even though sometimes we may not feel like it, or we feel like we have so many things to do. But remember that if we record it well, it actually saves a lot of rework at the end, later on, right?

And speaking about just now. I was laughing when you said that the requirements could be fuzzy, but the developers will build only one product that is very specific, right? Well, I don’t know now these days with all these AI capabilities, maybe the behavior of the product might have more variety with all the Generative AI. So I mean, looking forward maybe one day, even though the requirements is vague, the product is just one thing, AI will make it more multiplication in terms of features and capabilities.

Karl Wiegers: There’s another point I should make along that line. I mean, I know virtually nothing about things like ChatGPT and the wave of AI stuff going on. But in 1989, a long time ago, I wrote an article called The Laws of Computing, at least as I understood them to be back then. And law number seven, I think, is still worth keeping in mind: Artificial Intelligence is no substitute for the real thing. Yes. I think that’s still a valid law, you know. So these are tools, but, you’ve probably heard the saying that “a fool with a tool is still a fool”. Have you ever heard that saying? Yeah, it’s a pretty common expression, but I change it slightly. I say a fool with a tool is an amplified fool, and that’s worth keeping in mind.

[00:22:05] Practice #1 - Understand the Problem Before Converging on a Solution

Henry Suryawirawan: Nice, nice. So you mentioned just now about understanding the problem before converging on a solution. So this is actually the practice number one, part of the requirements foundation. So when you mentioned sometimes as a software development team, we are just given a task with a solution, right, and we have to just deliver it. But you invited people to actually start thinking about what actually the problem is.

So maybe from the situation that you encountered before, right? How can teams react when such situation coming that they are asked to build some kind of just a solution, just build this, right? And how can they actually capture the requirements from that solution better?

Karl Wiegers: Well, I think you need to think backward a little bit. If you’re presented with a problem, then I think it’s very helpful to do some root cause analysis. And you can build what’s called a root cause analysis diagram, sometimes called a fishbone diagram or an Ishikawa diagram. This is sort of a tree structure where you think backwards from the presented problem and try to understand the contributing causes. Like why is that a problem? And that’s the question we’re asking with this thought process is, well if that’s a problem or if that’s a goal that we’re trying to achieve, why are we not already achieving that goal? Or why is that a problem?

And you go through this through a couple of layers of asking why, and hopefully you’ll eventually get to some root causes that are contributing to the perceived problem. And those become actionable and we can figure out, well, how do we address those specific root causes? Or you might find through that thought process that what appears to be the problem isn’t really the real problem. Maybe that’s just a symptom of the real problem, or maybe it’s a tangential problem or just part of the real problem.

And you do the same kind of thinking if you’re presented with a solution and someone says, go build this product or this feature. And that’s where we start asking ourselves a question of why, if that’s the solution that you have in mind, what problem do you think that’s going to solve? What do you think it’s going to do for us? And why do we think that needs to be in there?

And that’s not really challenging. We’re not trying to challenge people or say you’re wrong when we do that. What we’re trying to do is understand. We’re trying to reach an understanding that we all agree on and can align towards a common shared understanding of the problem, a common shared vision of the solution with confidence that that’s the right solution for the right problem.

So just a matter of doing that backward thinking a little bit, I think is the right way to make sure that we’re not wasting our time or chasing something that turns out not to give us a good outcome. And a phrase that I’ve heard along the line as we were working on the book that I liked, if you don’t go through this thought process first, you might end up achieving project success. Hey, we built our solution. We delivered it on time and on budget, and it looks great. But product failure, because that solution isn’t actually solving the problem that you were hoping it would or exploiting the opportunity that you perceived. So we want to try to watch out for project success, but product failure.

Henry Suryawirawan: Wow. Yeah, I think that’s really, really important, right? Because many enterprise projects, right, even though they still stay through to the timeline, they stay through the budget, scope, and things like that, so they may not bring the kind of business impact or business value once the product is launched, right? So I think understanding the problem instead of just converging to a solution is really important, right?

And I think, in your book, you also give an example of a certain solution that was given. But I mean, in a hypothetical scenario, right, actually the team who has the understanding about the product is able to actually advise what kind of roadmaps or milestones that we can achieve. Don’t forget about that. So whenever you give an idea, maybe other people also have a better ideas, especially if the people are working with the product much, much closer than you are, right?

Karl Wiegers: Sorry, if I could just add to that just a little bit. Henry. This is an example of where people have different ideas of what the word requirements means. And you know, this conversation that we are talking about, this exploration of why – some people may not even view that as requirements, but that’s really addressing the business requirements that set the stage for everything else that we do when we build a solution based on technical requirements.

Henry Suryawirawan: Right. So, let’s move on to other practices. As you mentioned, there are 20 practices, but we won’t be covering all of them for sure. But I will just pick some from my interests.

[00:26:26] Practice #3 - Define the Solution’s Boundaries

Henry Suryawirawan: So the other thing that you mentioned about requirements fundamentals is that we need to define solution’s boundaries. In particular, what’s in scope and what is out of scope. So can you tell us maybe why it is important, especially to mention the out of scope part?

Karl Wiegers: Yeah, every project, every product is finite, you know. It has some things it does. It has some things it won’t do. And part of understanding our business requirements and our business objectives is to draw that boundary between what’s in and what’s out. And I saw that very nicely done at a company once who was building a website. And not only did they have a list of the things that the website was gonna let people do, but they also had a list of things that it explicitly was not going to let people do that someone otherwise might think it would. And that’s very helpful for people saying, oh, okay, well, we’ve decided it’s not going to provide this particular service. You might do that in the future, that’s fine. But for whatever release you’re talking about, we’re not going to do that.

And that helps you avoid the dreaded specter of scope creep where people keep adding in this and that. And, gee, wouldn’t it be nice if we did this? And how about that option? And there’s this new kind of credit card. We have to be able to take those now too. Well, at some point you have to draw the line or you never finish anything. And so I think understanding the boundaries between systems is helpful for release planning, for scope management. And also it’s very helpful to see where our system, our solution that we’re describing, fits into our universe.

And there’s some pictures you can draw to help you visualize these things. When we talk about requirements, you sort of naturally think about natural language text, and that’s a big part of it. But we also can draw all kinds of pictures, visual analysis models, to represent information that are really valuable. For example, a simple context diagram – which is not a new concept, it’s been around for decades – a simple circle with some boxes on the outside and arrows connecting the boxes to the circle. That helps you see what’s inside our solution, inside that circle, and what does it connect to on the outside. Those could be users, they could be other systems, could be pieces of hardware. That’s very valuable information to draw that boundary.

Another thing you can do is you can create an ecosystem map. And this is a valuable if you have a system that fits into an existing world, as most do, rather than being completely standalone. And perhaps the solution that you’re building now requires interfaces to some of the other systems that you already have in your universe. And maybe those are not just immediate connections, but they could be two or three levels removed. So there’s some passthroughs and communications and interfaces that get involved.

And by drawing an ecosystem map, you can more easily see how all of these things, all these pieces of systems in your world, fit together in a web and what the interfaces between them look like at a high level. And that helps you see dependencies. That helps you see where we’re gonna have to build some glue to connect these systems so they can communicate. Maybe you’re using some third party applications for security checks or for payment handling or something like that. So you can see how those things fit together. But we gotta draw those boundaries, because otherwise, you don’t know how to tell when you’re done.

Henry Suryawirawan: And speaking about scope creep, just now you mentioned, right? So I think by specifying clearly what is not in scope, we kind of like covered, okay, this is the things that we decided early on that we will not be building or we will not be including so that later on in the future we cannot have a room for some, maybe cheeky product leaders or executives to request something, right, request to add more stuffs even though maybe that request sounds so simple, but sometimes it’s not that simple whenever you try to build the system from scratch, for example.

So I think having the out of scope is really, really important. So for software teams out there, if you haven’t done this practice, please make sure that you define explicitly from the beginning what will be in scope and what will be out of scope. This is practice number three.

Karl Wiegers: And it’s helpful to also know what might be coming in the future. So maybe some proposed feature is going to be out of scope for now. But we see that as a growth direction, so we might add it in the future. And by being aware of that, you can at least build in hooks and, you know, the interfaces, so that it would be easy to plug in something later on to grow in that direction, but not today.

[00:30:47] Requirements Elicitation vs Requirements Gathering

Henry Suryawirawan: Yep. Thanks for adding that. So let’s move on to the next practice. This part is actually about requirements elicitation, but before we move on to the practice, maybe for some people who are not yet familiar with this term elicitation, right. Can you maybe explain and elaborate, how does it compare elicitation versus gathering requirements, which maybe some of us are more used to.

Karl Wiegers: That’s a good question, Henry. People do often talk about gathering requirements, but when I hear that term, I have a mental image of walking around with a basket picking flowers. They’re out there. You just pick out the ones you like and you put them in the basket, or hunting Easter eggs or something like that. In other words, gathering requirements to me, sort of suggests a collection process. And that’s definitely a part of it. You know, sometimes we do know of things that we are going to have to build or have ideas we can just put into the basket.

But the BA, the business analyst, isn’t just a scribe who just writes down whatever people tell them they want. Instead, the business analyst is a guide who’s leading this exploration of a requirements space. And so elicitation is partly collection, partly discovery, partly invention. And the BA is the guide for those processes.

So elicitation involves asking questions, for example, to see if people would find particular ideas or capabilities useful. So someone says, “Well, we want this.” “Okay, well, would it be helpful,” the BA might ask, “would anybody ever want to do that,” which is maybe an extension of this. And they say, “Hmm, I haven’t thought about that. Well, no, I don’t think anybody would want to do that.” Or they might say, “Oh wow, that’s great. I’m sure glad you thought of that, cause we didn’t.” So elicitation is a richer process. It’s more collaborative, not just a matter of collection. So I try to avoid saying requirements gathering.

Henry Suryawirawan: Yeah. So I think, there’s no situation out there where we can just gather requirements, right? I mean, we know clearly a 100% what we want to build and we just write it down, right? So I think that never, I think, at least in my experience, that never happens. So there’s always some room of ambiguity, some room of ideas as well from different people. So I think requirements elicitation is probably the proper term that you suggest here.

[00:33:01] Practice #6 - Understand What Users Need to Do With the Solution

Henry Suryawirawan: So let’s go to one of the practice, it’s practice number six, understand what users need to do with the solution, right? So just now we talk about understanding the problem, not converging to the solution. But in this requirements elicitation, you also suggest people to actually understand the need of the users with the solution. Maybe tell us more about this one.

Karl Wiegers: That’s one of my favorite topics: understand what users need to do with the solution, practice number six. But in terms of importance, I think it could be considered practically practice number one. In fact, one of the most important things I learned in my career as a software developer, is to focus not on features, not on the product itself, at least at this stage of requirements exploration, but rather focus on usage. What do people need to do with the product?

And that’s a much better approach to exploring requirements, usage-centric exploration as opposed to product- or feature-centric. Because by understanding what people need to do with the solution or certain parts of it, we can then put on our business analyst hat and we can derive the necessary functionality that has to be implemented to let people do that.

We can also prioritize at that higher level of abstraction that says, oh, okay, doing this task is more common or more important than doing that task. So what are all the bits of functionality associated with those two various tasks? That helps us prioritize at a little bit higher level of abstraction instead of trying to take 500 requirements or stories and figure out, well, which one’s most important. We can deal with complexity through abstraction that way through some hierarchy.

And that’s a very good example also of how to change that question. Remember, I suggested earlier that bad questions to ask around requirements are, what do you want? Or what are your requirements? A better question is, what do you need to do? What do you want the solution to do for you? Let’s not ask what the solution should have in it. Let’s ask what you need to do with this solution. So that’s where things like use cases and user stories can become valuable, and that fits at the level of what I consider to be user requirements.

The IIBA, the International Institute for Business Analysis, talks about stakeholder requirements as being kind of between business requirements and solution requirements. I don’t really like that term, because basically, as I see it, all requirements come from some stakeholder. So I’m specifically talking about user requirements, the things that various kinds of users need to do with the solution. And that to me is a really valuable way to both make sure that we include all the functionality people need to do those things.

And have you ever worked on a project or seen a project where people built functionality that someone said they needed, but no one ever used it? Have you ever seen that happen?

Henry Suryawirawan: Yes. I think it happened in some parts of earlier of my career as well. So we have to build a lot of features, but, I’m particularly not sure like how many users actually use that.

Karl Wiegers: Yeah, that’s very common. In fact, there’s data. People have tried to study in the software industry of what percentage of the features, however you define that, in common products never get used or rarely get used. And it’s over 50% in some of these analyses.

Well, you worked very hard building those features that nobody used. So I think it’s important to make sure we can focus our attention and our priorities on the things that we know people are going to use. And that’s why understanding usage patterns is a really important starting point for elicitation.

Henry Suryawirawan: Yeah, so in product, what some people refer to this is knowing the jobs to be done for the users that want to use the product, right. And I think also another important point, which you mentioned in the book is that, sometimes we can demo these features, the cool features that we just build to the users and they will be wowed and they seem to like it.

But at the end of the day, in the real world, they may not use it, because that solution, that features, maybe not solving the actual need, right? So they don’t have a use case in the particular situation that they would require the solution from the feature. So I think that’s really important, is knowing the user’s need instead of just building features and solutions over solutions.

[00:37:06] Software Requirements in Agile

Henry Suryawirawan: So let’s maybe touch on a little bit about the Agile world since you mentioned it earlier, right? I think in the Agile world, sometimes the practice of requirements is perceived to be different, right? So people call it user stories, writing in a card or post-its and things like that where the requirement seems to be just a one liner. As a user, I want this, so that, blah, right.

So maybe if you can touch on a little bit and advice for the Agilist people outside there. What is the better way of writing software requirements for the Agile software teams?

Karl Wiegers: Well, the first problem I have when this kind of question comes up is, what difference does it make? Because the developers need the same information to build the right software correctly, regardless of how they’re building it. They still need that same knowledge. And so I find the idea that requirements are all different on Agile projects, kind of silly, frankly.

Now, certainly there are differences between Agile and traditional projects in how they slice and dice these practices. You may do things in smaller increments. You may not write down as much, which works fine if you have short timelines, and things aren’t changing very rapidly, and people are working close together day to day, that’s fine. But it doesn’t change the nature of the information that people need to do the job. So I think whether you call them user stories or epics, that’s a mystique. That’s not substantive in my view. I think the idea of how you phrase user stories is valuable by saying, as a particular type of user, I need a certain capability, because of some justification, some rationale. That’s helpful to put it into a context that’s more valuable or more understandable so people can see the whole picture a little bit better. But I think really, whether you call them user stories or not is not important.

One concern I do have is that you can do user stories well or like anything else, you can do them badly. And if the user story becomes the only quantum of requirements, then really we’re trying to take all those different kinds of requirements knowledge that we talked about at the beginning of our conversation and turn them into user stories. And some of those stories might be bits of functionality, some could be full feature descriptions, some could be non-functional requirements like security requirements, say, or architectural issues. Some could be business rules, some could be data definitions. Well, those are all different kinds of information. And if the only container you create to store requirements knowledge is this thing called a user story, how do you tell those apart? How do you connect them? How do you see that, oh, here’s some pieces of data which really tie together multiple functional user stories. Or here are some business rules that apply to certain user stories, but not others. Or here are some performance requirements that affect certain user stories, but not others. So I think that taking any idea you have about something called a requirement and forcing it into a user story construct can be less than helpful at times.

Henry Suryawirawan: Yeah, so I think some people have always this misnomer, right? So whenever they find, for example, in the Agile Manifesto, it is said that value collaboration more rather than writing comprehensive documentations. So that’s the first thing. And people also seems to refer to user story as the only way of specifying requirements, because we want to be agile.

But I think what I find challenging in my day-to-day experience is that sometimes these are not enough, right? Just one-liner of, as a persona, what kind of things they wanna do or what kind of business value it brings. It’s not sufficient for developers to actually write the software, right? We need better specifications, we need better business rules, validations and things like that. So I think sometimes this misnomer, I think is something that is frustrating.

[00:40:59] Requirements & Effective Software Development

Henry Suryawirawan: And what would you advise people to do then? I mean, if you have a user story, do you also capture some business requirements somewhere in a different templates, so that you can actually correlate between multiple user stories and a business requirement? So maybe if you can give some practical tips here for people to probably improve their software development process.

Karl Wiegers: Yeah, I think that’s a good idea. In fact, the business requirements aspect, which includes multiple kinds of information, the idea of business requirements. And you can see a template for a vision and scope document from my book. And both the Software Requirements Essentials and the Software Requirements book contain a template for a vision and scope document. And that’s just a container for business requirements. So while I call these things documents, they can take many different forms. They’re just containers for knowledge.

But those kinds of things really don’t have anything to do with how you’re going to build the product. What development life cycle you’re following? What are your business objectives? How are we going to measure whether we’ve achieved our business objectives? What are our constraints that we have to work within? What’s our vision of the product? What are the limitations that we’re going to have as exclusions, like we talked about earlier, that aren’t going to be there? These kinds of things really don’t have anything to do with how you build the product. So I think they, they lay a solid foundation for any kind of project that you might be working on.

We need some way to tie together the different pieces of information that are related. Now, if you have a whole bunch of user stories, and they’re logically flat as opposed to hierarchically grouped in some fashion, that’s harder to manage, I think. If you have a big project and you have 500 user stories, and no really good way to group them, then how do you manage that? It becomes a harder problem than if you can group them into features and see the relationships there and prioritize at that level and so forth.

So I think it’s important to identify for every project, really, what’s the best way for us to document at an appropriate level of detail at an appropriate time, to record the information that’s essential, to see how it fits in with all the other information we have, to see the things that span sets of use cases or user stories and so forth. Every project should decide how to do that. And the point is not to be Agile. People sometimes get confused about that. Who the hell cares if you’re Agile? What they care is if you’re effective. That’s much more important.

So I think the idea that, well, we can’t do that, because it’s not part of Scrum, that’s just silly. You’re not trying to be Scrum, you’re trying to be an effective software development team. So let’s keep that goal in mind and then let’s choose those practices, those kinds of containers, those levels of detail, those role assignments and all that. Let’s choose the approaches that are going to help us be as effective as we can be to build the right solutions. And if that means that you violate some Agile rule, I can tell you the Agile police are not going to come after you, that that’s not a threat.

Henry Suryawirawan: Yeah, that’s a funny joke. Definitely. But I think that rings really true, right? So the point here is to be an effective and productive software development teams. And also you minimize a lot of rework as well, right? Especially if you didn’t specify the requirements clearly and the developers build something, but turns out, oh, actually this is not what I want. So the developers come back, fix it, and the cycle goes on and on.

So I think you mentioned about templates as well. So don’t forget for people who want to read the book, Karl also put a lot of links to the websites of the book, where you can find a lot of templates. So these templates doesn’t relate to any software development methodology that you use. You can use it in any kind of projects, so make sure to check it out as well.

[00:44:39] Rework

Karl Wiegers: You’ve mentioned, I think, twice now, Henry a very, very important word that I’d like to talk about for just a minute, and that word is rework. You know, everybody wants to be more productive. My first question then is, “Okay, well, why are you not already as productive as you want to be? What’s the problem?”

And very often the problem is that teams end up spending more time than they expect doing unplanned rework, doing over something that they thought was already done. And software teams that measure how much of their total effort is spent on rework can get very scary numbers. 30% or 40% of your total time might be spent doing things over. Now, some of that’s just the nature of what we do. Okay. It is an imperfect process. Things change. People learn more. That’s fine. So we have to expect some of that. But you raised the point a couple of times, and I’m very glad you did, because I think people should be actively choosing approaches and techniques to try to avoid the unnecessary rework of having gone this way when you should have gone the other way.

A lot of the effort we invest in requirements is not to pretend that we can get it all right the first time and write it down once, and then just go build the product. It’s to lower the risk of having to do things over later on when we planned to do something else instead. So a lot of how much you choose to do around requirements really boils down to controlling the risk of wasting time on unnecessary rework. So I just wanted to pursue that thought a little bit. Thank you.

Henry Suryawirawan: Yeah, thanks for emphasizing that. So I think for software development teams who haven’t really measured the amount of rework that you have to do, I think maybe you can do that exercise. And maybe you’ll be surprised by the percentage. Just like what Karl mentioned, it could be 30%, 40%, or even 50%.

[00:46:20] Practice #19 - Establish and Manage Requirements Baselines

Henry Suryawirawan: So in your book, you break down the practices into multiple areas like requirements elicitation, analysis, specification, validation, and management. We won’t be covering all of them. Let’s maybe fast forward to the last part, requirements management, which I find, maybe, it’s also important to cover here in our conversation. The practice number 19 is actually talking about establishing and managing requirements baselines. So first of all, what is baseline? And the second thing is why it is important to establish a baseline?

Karl Wiegers: Very good question. And that’s one of those things that could maybe turn off Agile people. It’s like, oh, baseline, well, that sounds like old talk to me. We don’t do that stuff around here. Well, of course you do. A baseline is really just an agreement. It says that for some body of work, whether that’s a single development iteration, or a release, or an entire project and product, we were reaching an agreement among the key stakeholders about what’s going to be in that delivered piece of software. And that’s all a baseline is. It’s a reference point.

And as we’re going along through our requirements planning and our project planning, again, regardless of whether you’re doing agile or traditional or some hybrid approach, we’re iterating on that until we decide, okay, this is what we are settling on as being stuff that will fit in the size of the box that we have for this upcoming work. We’re making agreements. We’re making commitments at that point to other people, so they know what to expect, whether that’s for 1% of the final product or a 100% of the final product. So you do that kind of thing, basically, on an Agile project, you do it iteration by iteration by saying, okay, these are the stories that we are planning to implement in this iteration. That’s a baseline. It’s not more complicated than that.

And that’s your reference point for change. I mean, change happens. We have to adapt to change. And that’s actually practice number 20 is to manage changes to requirements effectively, affects every project again. And so what we’re doing is we’re saying when a change comes along, how does that affect our current and future plans? Does it go into a current baseline that’s under development? Well, that means either it’s going to take longer to finish that implementation, or something has to go, something has to wait. There’s no right answer, but we have to make those decisions. And so a baseline becomes the reference point for making those changes, becomes a reference point for tracking our status toward achieving our objectives for that development cycle.

And I think reaching that clear agreement is a lot more valuable than saying, well, here, go do some stuff and call me when you’re done and I’ll let you know if you’re really done. That doesn’t work so well. So that’s all we’re talking about when we say define baselines, and there are different ways you can do that, depending on what life cycle you’re following. But I think it’s important for every project to do that in a thoughtful way.

Henry Suryawirawan: Right. So baseline is nothing but like an agreement, right? A set of commitment that the team is delivering. It could be per iteration, it could be per release cycle, or it could be anything. But the key thing here is that you need to set a baseline, right? Because otherwise, you can’t keep building things dynamically day by day. So you need a reference point.

[00:49:17] Practice #13 - Prioritize the Requirements

Henry Suryawirawan: And speaking about, coming to the baseline, you also need to prioritize the requirements, right? So maybe if we can touch on a little bit, how do we actually prioritize requirements that we have, which could be a lot and committed to a certain of baseline. So how can we prioritize better? Maybe that’s the question.

Karl Wiegers: Well, there’s certainly a lot of articles written about different ways to prioritize requirements, different techniques that are valuable for it. And we have a practice on that. Practice 13 says prioritize the requirements. And I think teams should prioritize things in as simpler a way as they can. If we can have a conversation and a handshake to agree on what we do first and what we do later, that’s great. Sometimes you have to take a more analytical approach. You have to, maybe, get out a spreadsheet and use various spreadsheet tools, some of which I’ve developed myself that help you think through the process of deciding which ones are more important or urgent to do.

There are a bunch of questions that you can think through. And I think every team should decide, well, how do we make those decisions? What factors are important to us when we try to make priority decisions? And you might consider things like the customer or business value it would deliver, how it would contribute to the project or the development iteration’s goals. Who requested that particular feature compared to some other feature? How often is each one likely to be used? What’s the cost or time or technical difficulty associated with implementing it? So there are a lot of factors – that’s just a handful of them – that you might want to consider.

So I would say, first of all, each team should think about what is its decision making thought process about priorities. Who even makes those decisions? Let’s figure out who’s going to have the authority and the responsibility for making those decisions. There are a lot of ways people have decided to do this. There’s planning poker. And there’s things like, okay, here’s a hundred imaginary dollars. How do you allocate those? Do you try to rank order them? Which is fine, if you have 15 items, not so fine if you have 300.

So there’s a lot of techniques. I wouldn’t say there’s any one particular technique that always fits in every situation. But the most important thing is for people to consider early on who makes the decisions. How do we make those decisions? What do we think through? What do we consider when we’re going to do that? Because that drives the sequence in which you build the work that you have from your backlog of pending possibilities.

It also helps you make decisions when we’re considering changes. Okay, something comes along, new feature. Well, how important is that compared to these other things we plan to do the next development cycle. Does one of those wait because this is more important? Those are all part of the thought process. Very important part. And that’s something I’m glad the Agile teams have emphasized more. But still, every project team has to go through those same considerations.

Henry Suryawirawan: So what I hear you say just now is that the importance of knowing the key factors, right? So the teams need to be able to put down the trade-offs. What are the levers that you want to think about, so that the teams can make decisions uniformly, right? It’s not like one day you use this kind of a trade-offs. The next day is different. The next day is, maybe, the highest person said something, then you have follow. So you have to, probably, define the key factors that you want to include in the prioritization so that we can use it over and over again.

[00:52:22] Practice #20 - Manage Changes to Requirements Effectively

Henry Suryawirawan: So, speaking about managing changes, this is the last practice that you have, practice number 20. This is a real world, right? So things happen. Software requirements change, sometimes in the middle of the iteration. That means also changing your baseline, right? So how we can manage changes to requirements effectively. I think for people, who probably most likely, are going to encounter this over and over again in their software development career.

Karl Wiegers: Well, the first thing every team needs to do is to adopt, I think very early on, a change process. And again, that’s something where some people might say, hey, well, you know, we don’t have a process. We just gotta get stuff done. Well, the process is just how you get stuff done. That’s all it is. It doesn’t have to be fancy. But it has to be effective.

So we have an example in the book of a flowchart, a process flow from an Agile project that shows the participants processing a change request and making decisions of what its impact going to be. You know, we need some information to be able to handle a change. You need to know what’s being requested. You need to do a little impact analysis so you understand what it’s likely to cost and how it’s going to affect other pending or completed work or other systems. When would that change be worked into the development activities? Who makes the decisions? Again, that’s a very important part of it, is who makes the decisions and how do they make those decisions?

We actually address that very important issue in practice number five, which is to identify the empowered stakeholders. Or decision makers, rather, identify the empowered decision makers. So having a process then makes it more systematic and less ad hoc. We need to know who receives the change request and how it gets processed. That can be very simple. One person might do the whole thing. That’s fine. But if you’re building a Boeing aircraft or something like that, there’s probably gonna be more than one person involved in making a decision about something to change. So everything in between a tiny little app you’re building yourself and building a huge complex systems project.

So adopt a process. And, you know, it’s interesting because I’ve worked for Kodak for many years before I started my own software consulting company, Process Impact. And my last job at Kodak, I spent six months in the development group for kodak.com, their website, and that was before the web had taken over everything in the late nineties. And it was a very fast moving, very Agile kind of environment. And I came in for six months to lead some process improvement activities for this group. And one thing that we put into place that was extremely helpful was in fact a defined change process, which really helped them bring order out of the chaos so that they knew what to work on, when, and why. And the people in the group had the exact correct attitude. They viewed this process not as a straightjacket, but as a structure.

So people shouldn’t be afraid of process. A good process helps you get things done more effectively, more consistently, more reproducibly. But if the process doesn’t work, people will find a way to get around the process, and they probably should. So that’s a clue that, hey, maybe your change process needs some tune up.

Henry Suryawirawan: Right. So regardless whether you are an Agile practitioners or more traditional practitioners, change process, I think, is really crucial, right? Especially as we all know, in any software project or software product, you will have changes. You will have some kind of new ideas, new feature changes, right? Or new bug, somehow critical happening in your product. Then the team will need to handle it, as soon as possible. So having the change process, the prioritization technique. All this will also play a part into building a more effective software development team.

[00:55:54] 3 Tech Lead Wisdom

Henry Suryawirawan: So Karl, I feel that we will not be able to stop talking, right, about requirements, because there are so many great things that we can cover here. But unfortunately, we have to wrap up pretty soon because of the time. But if you still remember last time, I have one last question at the end of our conversation, which is called the 3 Technical Leadership Wisdom. I will ask you the same question this time. So hopefully, you will find new ones based on this new book. Or maybe if you can refer to the previous ones, that’s also fine. So maybe if you can share your three technical leadership wisdom for this episode.

Karl Wiegers: Okay. I think that’s a very good question. And the first thing I would suggest, as you listen to this podcast, as you read the Software Requirements Essentials book or other books, if you watch the videos that we’re putting out (I just recently, actually just today, started a series called The One Minute Analyst, where I’m posting on my YouTube channel some very quick videos on these topics). Don’t feel bad if you don’t already do all these things on your project. Okay?

That’s the first piece of wisdom is, maybe you look at all these things that you should be doing and sound like a good idea, and you say, oh, we’re not doing that. You know, we’re, we’re failing. No! Don’t feel bad if you don’t already perform all those practices on your project. That’s fine. That’s where everybody starts.

The second one then would be as you go through this information, try to identify those practices that you think would add the most value to your project. Look for opportunities to try them and situations where some of these practices or techniques might yield better results for you, might solve points of pain that you’re currently experiencing, or might keep you out of trouble later in the project by reducing risk as we mentioned earlier. So that’s the second point, is look actively for practices that you think would be worth trying.

And finally, don’t try to do everything at once. People can only absorb change, organizations can only absorb change, at a certain rate. And remember that there’s a learning curve that’s gonna slow you down a bit as you try to figure out how to make some new method work for you and for your team. But over time, these new ways of working based on these practices that you feel confident or worth trying out, those will just become part of your business analyst tool kit. And I’m pretty confident you’re gonna get better results once that’s happened.

So those are my three pieces of advice. Maybe wisdom, but at least advice.

Henry Suryawirawan: Thanks for sharing that. I still remember the last wisdom that you mentioned, right? Don’t do things at once. It’s like your ultimate pearl from your previous book, right? So we covered that also in the previous episode. Thanks for sharing this. So we all want to be, better, right? Especially knowing all this information from this podcast or from YouTube videos, books, and things like that. But don’t feel bad if we haven’t actually done any of these. So we can always start something small, do the things that bring us most value or solving the biggest problem for us, right? So thanks for reminding that.

So Karl, for people who want to learn more about the book or maybe resources from the book, is there a place where they can find you online or the book resources online?

Karl Wiegers: Definitely. I have two websites. My personal website is karlwiegers.com. And that’s I before E except after C. Karl with the Wiegers. My business website is processimpact.com because my company is called Process Impact. And we have a website, a very simple website, for the book. The book is Software Requirements Essentials and the URL is softwarereqs. That’s software r-e-q-s .com. And so people there can see these videos, they can read a sample of the book, they can see the list of 20 practices. We’ve talked about several of them here today, but there’s a lot more there. And there’s also a bunch of checklists and document templates and spreadsheet tools and other work aids that we’ve made available there. So there’s a lot of useful stuff at softwarereqs.com.

Henry Suryawirawan: Right, and I will suggest people to also read the book, right? It’s very thin. It’s not thick like compared to the Software Requirements book. And I find any kind of role in software development team would benefit by reading this book, because then you will understand the essentials of software requirements, which most likely is one of the biggest factor of building the successful product. So thanks for sharing this and thanks for writing this book.

So Karl, really a pleasure to have a chance to talk to you again. And hopefully the book is becoming successful to transform people to write better software requirements.

Karl Wiegers: Thanks, Henry. It’s been a pleasure to have another stimulating conversation with you. And if we can send people away with one final message, that I’ve believed for a long time, because I’ve been interested in requirements for a long time, is that if you don’t get the requirements right, it really doesn’t matter how well you execute the rest of the project. You will fail.

Henry Suryawirawan: Wow, really well said!

– End –