#99 - Better Software With Acceptance Test-Driven Development - Kenneth Pugh

[00:01:42] Episode Introduction

Henry Suryawirawan: Hello, my friend. Welcome back to the Tech Lead Journal podcast, the show where you can learn about technical leadership and excellence from my conversations with great thought leaders in the tech industry. And this is the episode number 99. I can’t believe that we are just one episode away from episode 100! If this is your first time listening to Tech Lead Journal, make sure to subscribe and follow the show on your podcast app and social media. And for those of you who want to contribute to the creation of this podcast, support me by subscribing as a patron at techleadjournal.dev/patron.

My guest for today’s episode is Kenneth Pugh, also known as Ken Pugh. Ken is an acclaimed author and thought leader in acceptance test driven development (ATDD) and behavior driven development (BDD). In this episode, Ken explained in depth the concept of acceptance test and ATDD. He first described what an acceptance test is, why it is beneficial to deliver better software, and why we should invest our effort to automate it. Ken also touched on a few other important concepts, such as the testing triad, test pyramid, user acceptance test, and table driven specifications. Towards the end, Ken shared some advice on how we can start implementing ATDD.

I really enjoyed my conversation with Ken, deepening my understanding of acceptance test, the insights Ken shared regarding where acceptance test resides in the test pyramid, and his view about user acceptance testing. If you also find this episode useful, I would appreciate if you share it with your friends and colleagues who you think can also benefit from listening to this episode, it is my ultimate mission to spread this podcast to more people and I need your help to support me towards fulfilling my mission. Before we continue to the conversation, let’s hear some words from our sponsor.

[00:05:31] Introduction

Henry Suryawirawan: Welcome everyone to another new episode of the Tech Lead Journal. Today, I have with me a guest named Kenneth Pugh or Ken Pugh, in short. He’s a very experienced thought leader out there. So he has written a number of books and specialized in acceptance test-driven development and behavior-driven development. So as you can guess, we will be talking a lot about ATDD and BDD today. Ken wrote some books. One of it won the 2006 Jolt award winner, which is the “Prefactoring” book. Followed by his latest book called “Lean-Agile Acceptance Test-Driven Development”, and he’s also a co-creator of SAFe Agile software engineering course.

So Ken, really looking forward to have this conversation with you. I’m going to learn a lot about ATDD and BDD, I guess.

Kenneth Pugh: I’m glad to be here and chatting with you.

[00:06:16] Career Journey

Henry Suryawirawan: So Ken, in the beginning, I always love to start my conversation to understand about your background. Maybe if you can share your career journey, any highlights or turning points in your career.

Kenneth Pugh: So, I graduated with an electrical engineering degree. They actually did not have computer science much in those days. I went into digital logic engineering, and it turns out after about five years, the Intel 8008 came out. Not the 8080, but the 8008. And I suddenly realized that the future was not in digital logic, but in software. So I actually made a switch at that time. I went into software. I did some programming on radar systems, on long baseline interferometry and other stuff. Was out in the Marshall Islands for a while. Then when I came back, I started my own consulting firm and did custom programming. Of course, those were when the PCs were just coming out. So your programming was on a PC. No network, no nothing. So it had to fit in 64K of memory. All the programs.

So then I was doing custom programming. Got into training in the C language, and then from there have been doing training, wrote some books, wrote a couple of books on C, and then progressed to more consulting. Agile came out around 2000. I was doing some of the tenets that were actually in XP programming already. I was writing tests for all my code. In C code, there was no C unit, but it was tested, writing the test for the code first. And then went into agile consulting, agile training, joined with Net Objectives for a few years. I was doing design patterns, test-driven development. About 12, 15 years ago was when I started creating the course on ATDD, BDD and then eventually that turned into the book.

Henry Suryawirawan: Thanks for sharing your story. Looking back, I think it’s a very long journey, right? If you can share, what made you decide that software is the future? What things that made you believe that actually software is the future?

Kenneth Pugh: Well, many years ago, I actually had spent over a couple of months designing a digital logic circuit. And then when the 8008 came out, I realized that I could have done the same thing just in firmware without trying to hook up all the digital logic gates and everything. And I said, guess what? It’s going to be software. Now, the software wasn’t big in those days. In college, I had programmed an IBM 360. So I was familiar with assembly language and Cobol and everything. And I realized that the digital logic was being taken over. You could write a firmware program and do the same thing instead of laboriously drawing on a block diagram.

Henry Suryawirawan: I see. So, in the last few years, I think you specialize in ATDD (Acceptance Test-Driven Development) and BDD, right? Although in your early years, definitely, you started from programming, practitioners of XP, Agile and all that. Let’s focus today, maybe predominantly on acceptance tests.

[00:09:30] Acceptance Test

Henry Suryawirawan: So you wrote this book, “Lean-Agile Acceptance Test-Driven Development: Better Software, True Collaboration”. But first of all, I think let’s clarify the definition. So what is acceptance test? Because I hear this term being used multiple times. Sometimes means differently to different people. So maybe if we can start from there, that would be great.

Kenneth Pugh: Okay. So my definition of acceptance test is any test that a system must pass in order to be accepted. That can be provided by the customer, the users, the testers, cause the testers still play a big role, or anybody else. And if you can’t ship a system without these tests, well, they are the acceptance tests.

Henry Suryawirawan: Wow. It’s very concise. Actually, I was struck by your definition of the test that actually makes the system accepted. I know a lot of people these days, especially maybe in the startups, we have tests, but we just deploy it, anyway. What does it take for people to realize that, okay, you really need to have these kinds of tests before you can actually accept the system?

Kenneth Pugh: You’ve got a couple of things here. First, if you know what you want the system to do, as opposed to you’re just experimenting. And it’s hard to create acceptance tests if you’re experimenting with a system. It’s like you just want to try something out, prototyping it. So we’re not going to be writing acceptance tests for those sorts of things, cause they change all the time. But once you’ve decided in your experiment that this is what you want the system to do, then writing the acceptance tests absolutely clarifies the requirements. You don’t get into this issue of, well, should it work for 1,000 or 900? You have a test for it. Either the test says 900, or the test says 1,000.

By documenting that test and it actually then becomes in the term executable specification, if you will. It means that now our description is the test, as opposed to a set of requirements that, well, that’s nice, but what does it mean? How are we sure that we’re meeting the requirements? And that misinterpretation of the requirements is probably the biggest thing that goes back and hits people. It’s like you get to the end and the testers find a defect. And the developer said, “No, the requirement said this,” and the tester said, “No, the requirement says that”. Why didn’t you guys agree ahead of time what the requirement said?

Henry Suryawirawan: I think that’s a very important key point, right? Because you mentioned requirements sometimes is misunderstood. Sometimes it’s vague as well. And sometimes it’s just one liner. Okay. Build me this. You bring a point in your book that you mentioned that a testable requirement should have acceptance tests associated with them. So I think this is also a key discipline. When you define requirements, you always have to define it in such a way that it is testable. If you can elaborate a little bit more on this point.

Kenneth Pugh: Okay. So there’s a concept of a testable requirement. How do you know the requirement is testable? By writing a test for it. If you cannot write a test for a requirement, then the requirement is untestable. If a requirement is untestable, how do you know the system is doing it? Why are you asking for that requirement if there’s no way we can tell this system is actually performing that behavior?

Henry Suryawirawan: Yeah. That is actually true. If you define a requirement in one liner but then leave other people to test. If at the end you think the software is not acceptable, I mean, at the end, it’s your fault as well. So you don’t actually specify the requirement that can be tested in a way.

There’s a point where you mentioned that acceptance tests, if done properly, can be an authoritative and reliable source of what the software should do functionally. So I think, especially, you mentioned about executable specifications, if everything even is defined by using tests, I think that’s a good way of getting the reliable source of truth about all the requirements that the software does.

[00:13:39] Acceptance Test Benefits

Henry Suryawirawan: So we know about the definition. I think some people also know about the benefits. But can you elaborate what are the benefits of acceptance test?

Kenneth Pugh: Okay. So the main benefit, it’s always based in your context. If I go into some place and I ask them, “So how many times does something come back from the testing environment or from production back to development with a defect?” And if the answer is 0.01%, I’m going to go, “Well, there’s not much I can do for you. I mean, we can spend some time, but you’re pretty good.” If the answer is 1%, I’m going, “Well, how much time does it take you to solve the defect?” If it’s starting to creep any more than that, there are places that are, I call them, a loop back. A loop back is when you send something out and it gets sent back with a defect that you should have thought about before. As opposed to feedback, which is you send something out and you learn something new to help improve the product.

If you can cut your loop back rate down dramatically, then you can flow that business value through your developmental value stream much quicker and get things out. So that you’re actually always working on new stuff, as opposed to fixing defects. And in fact, that’s the way I usually sell it to developers. I go, “Hey, guess what? You’re getting your requirements in a testable form. Your job is to write an implementation that passes those tests. Now you’ll still maybe have some communication and collaboration because maybe the tests aren’t quite clear and so forth. But once you pass the test, you are done.” If we are able to describe as much as possible requirements and tests that would’ve normally been done in a post implementation phase, that means that you won’t have any defects to fix. Do you like fixing defects? Do you spend your life just going, “Oh man, I just want another defect I can fix today.” So that’s how I sell it to a development team.

Henry Suryawirawan: Well, some people with wrong incentives might like fixing defects because some people are incentivized by number of defects found. But I mean, joke aside, I think I get your point that if we can reduce the number of loop back or rework, or something that goes back from your value stream, right? From the last back to the beginning, again, I think that can cut down a lot of inefficiency.

[00:16:18] When to Write Acceptance Test

Henry Suryawirawan: And you mentioned, I think, that if the product team or the developers and testers actually write acceptance test prior to the implementation, that actually can double the productivity. I think some teams I observe as well, after the requirements, maybe it’s a little bit vague and then they will have a phase within the same sprint, testers write the test cases. So tell us more, what do you think about this practice? How can we change it?

Kenneth Pugh: Okay. So it is a change. Because in many instances, it’s like developers just want to take their code and throw it all over the wall to the testers. Even if they’re on the same team, I’m going to call it “it’s a mini wall”. I’ve seen too many instances where, all right, the developers get a story done and they get it done at 4:45 on the last day of the sprint. And the testers then have 15 minutes to test it, or they have to work the weekend and everybody’s after them because the story isn’t done until it’s tested. And then, of course, what happens is this breaks into behavior going, “Oh, well, let’s create a testing story for the next sprint. So at least we’re saying we’re done with our development this sprint, and the testing can go next sprint.”, which, of course, obviously is really opposed to the idea of agile development.

And so the point is that if we and the testers and it takes just a couple of days to switch this thing around, you’ve got to be able to pull something off the backlog, spend an hour, two hours. And if it takes any more than that, you have a complicated story. But spend an hour, two hours creating all these acceptance tests and then start to implement it. Now, developers run the acceptance test while they’re implementing, or once they’re finished, depending on how much of the application they need. And then the tester’s job at the end is, “Okay, let me run it through on my environment if necessary,” or “I’m going to spend my 15 minutes doing some exploratory tests to see if there is some unintended behavior that we have not described by all the acceptance tests.”

What this does is it puts an entirely different spin on things. The number of defects that a tester has to report for purely the functional part, I mean, the things that when you looked at the requirements, it’s like, “Okay, this is the way it should work. And yeah, we’ve got a higher limit here, and if it goes above that limit, we should get a message that gets put up.” And it’s like, “Oh well, that’s an edge case.” That’s what testers do. Now, let’s write the test for the edge case and developers can make sure that it works that way. If the application is supposed to have a behavior for both good and bad input, we can write all those tests for it. And then the testers can be spending their job looking for all the unintended things. What happens if we do, instead of doing it ABC like we thought, we do ACB in our workflow? Does something break? It puts an entirely different spin.

And to just give an example, one of my things that I asked one team four months later to give me a report how well it works. And they said, the team, the lead developer, and lead tester are much more mellow. Because they’re doing testing throughout the sprint. It isn’t crammed in the last day and it creates the “we are the team” feeling. It isn’t us versus the testers. It’s everybody’s in it. They just found that their defects went dramatically down. Yeah. Do things slip through? It’s going to happen. But when the number of defects is like, you can use your fingers on one hand to count them, you don’t need to have a JIRA board to take care of these. It’s like, “Okay, let’s go fix them.” Or we’ll decide that, “Well, okay. Nobody’s ever going to do that one. So let’s not worry about that.” That’s a case that shouldn’t show up in normal stuff.

Henry Suryawirawan: Thanks for sharing the story and the impact that people have done by following your advice. I like what you said that the testing actually happened throughout the sprint, not at the last one week or few days before the sprint and everyone is cramming and waiting when can we deploy this feature?

[00:20:55] The Triad

Henry Suryawirawan: So you mentioned that in the beginning, the team should come up with these tests, spend one, two hours. You mentioned that the people that should be involved in creating tests, which is called the triad, some people call it three amigos. Can you elaborate what is this concept, the triad? How should people implement this?

Kenneth Pugh: So I call it the triad because that represents three perspectives, not necessarily three people. The three amigos tend to be three people and everybody wants to be Steve Martin. So you get into all those, “Ah, I’m Steve. No, you’re Steve.” So I call it the triad and three perspectives. The customer who is providing the requirements. The developer who’s implementing the requirements. And the tester who is critically analyzing both the requirements and the implementation. So they get together.

Now I give three perspectives. The customer’s perspective. It could be the product owner. It could be a business analyst. It could be a subject matter expert. It could be all three of them could be representing that perspective for particular story. So that’s why it’s not just three people. It’s the three perspectives. They meet with the developer, with the tester, and explain what the story is. They work together to create, in BDD terms, we call it a scenario. Just a given-when-then scenario of what the state is, what the action or event that occurs and then what should be the output or the new state. And they work on these. While they’re doing it, they’re defining domain terms. They’re defining ranges, out of range stuff. And they’re defining two other things, which is what I add to the normal given-when-then, which are business rules. The business rules, which for some systems take up 80% of the code, or at least 80% of the work.

For example, for an ATM, everybody has the ATM. Given I have some money in my bank, when I go to my ATM and withdraw $20, then I should have $20 in my hand and my account should be $20 less. Well, it turns out that’s nice, and that’s a simple thing. But there are actually about, in the banking industry, about 150 business rules on whether you can withdraw that money out. How many times have you withdrawn that money out? How much money have you withdrawn? So forth and so on, and I will not elaborate them all here. And it’s really that simple flow. That’s easy to automate. But it’s adding in those 150 business rules that need to be clarified precisely. If you’re banking, if you’re an investment firm, you’re letting your customers buy and sell stocks, you better have your business rules right, or you’re in trouble with the Securities Exchange Commission. So in addition to just the normal given-when-then of what I call a flow scenario, I emphasize business rules. Here’s some input in my business rule. What’s the output? Here’s how much money, how many times I’ve withdrawn over how many days and so forth. Given all these inputs, can I get my money out? Yes or no. And those become very elaborate. I have seen business rules that have over a thousand different combinations. Each one is unique and has to be done. There has to be an answer for it. It isn’t like an algorithmic sort of thing.

And then the third thing is our domain terms. This is a key where ATDD meets DDD, Domain-Driven Development. Because during this collaboration, we’re going to come up with those domain terms. Oh, balance, currency, so forth and so on. It’s critical that you get your domain terms right. Oh, for example, balance. Yeah. That’s what you all see in the little ATM sort of examples. But is it your current balance? Or does it include potential withdrawals that are going to be posted in the next two hours and so forth and so on? And we need to absolutely clarify these things to make sure it’s right. There are some cases where, oh, you take Facebook or something like that. Doesn’t matter whether you get this sent out to 5,000 people or 4,800 people. Now that’s an instance of, yeah, you can write acceptance test for it, but you don’t have to be quite as picayuney. A lot of my consulting is with financial, insurance, investment firms, and medical firms where the business rules are critical.

Henry Suryawirawan: Thanks for emphasizing all these key points. So maybe if I can summarize, you mentioned that the test should have the flow. Of course, the scenario given-when-then kind of thing. You have the domain terms. So I think in DDD, we call it ubiquitous language, where you use the terms that business people use. And as much as possible, maybe you should be able to see it throughout your code, throughout your test cases and all that. And business rules. I think you emphasize very clearly. Depending on the domain and the industry, of course, there will be some industry that has complex business rules, maybe elaborate business rules as well.

[00:26:31] Is Doing ATDD Expensive?

Henry Suryawirawan: You mentioned financial technology, healthcare and all that. But many people don’t work in this industry. So they think writing ATDD or acceptance tests in the beginning is expensive. Or maybe they don’t even understand all the business rules that are available and they’ll just leave it to the team to decide. So what is your counterargument to this ATDD is expensive to implement?

Kenneth Pugh: Well, I’m looking at it as we’ve got the user, we’ve got a customer. If the customer cares about the details of how something works, then we ought to create these acceptance tests. If all they said was, “Oh, I just want you to withdraw money out of an ATM.” We’ll let the team decide how to do it. You can write acceptance test though. I have an article on this about the acceptance tests vary in the domain, just as you’re saying. For example, if it’s a video that we’re doing. Suppose you have a video website and we want to write an acceptance test for video. Well, there’s not much we can accept. We can say we don’t want it to pause. We want it to start pretty quickly and I want to be able to forward and backward on it or something like that. That’s a very high level behavioral thing. The developers have a lot of work to do to implement that. And our acceptance tests really are just the external behavior and there’s a lot of internal behavior whose result is reflected in the external behavior, but which can’t be specified other than in those gross terms. So it’s perfectly fine if you’re in a situation like that, that you can specify just a gross external behavior and leave it to the developers to write an implementation about that. But it’s still good to have the acceptances cause then you sort of have an idea of when you’re done.

Henry Suryawirawan: So I think that’s a very critical point. Know when you are done. Because if you don’t have these tests or specifications, you probably don’t know when you’re done. So I think that’s really important. You mentioned that, actually, it doesn’t cost a lot more even if you write it in the beginning versus writing it at the end. So it literally takes the same effort, probably. But the impact, right? So you minimize the number of defects, number of rework, number of missed clarifications that need to happen between developers and the customers or business owners or product owners. So thanks for emphasizing that.

[00:28:56] Acceptance Test & Test Pyramid

Henry Suryawirawan: Maybe we can go into one level technical now. There are so many different types of tests in this technology world these days. You have unit tests, component tests, acceptance tests itself, API tests. So where does acceptance test actually get categorized? Is it the same all over the place or is it different? Maybe need your clarification here.

Kenneth Pugh: Oh, that’s an excellent question. Okay. We have all these tests. In fact, oh man, unit test, integration test, end-to-end test and so forth and so on. What’s a unit test versus integration test? It’s like, oh, is it a single method? Well, I go along with Google. Google came up with a concept. If you have a test pyramid, and at the bottom of the test pyramid are small tests. And in the middle are medium tests and at the top are large tests. So what you want to do is have a lot of small tests, less medium tests, then it’s very few on the end-to-end tests. Now we don’t get into an argument of whether it’s an integration test or unit test, because is it a small test? Does it run pretty fast? Regardless of whether it includes 32 classes or not, if it’s fast, it’s small. And so therefore, we’re not looking at trying to make these discriminatory things of, “Oh, it’s an integration because we include two classes or this and that.”

On this level, the three types of things that I’ve indicated, the flow, the business rules and the domain terms, well, in reality, the business rules and the domain terms actually are small tests. They go against either single class or maybe even a single method in the case of a business rule, although they involve more things. So they’re pretty fast. So we can’t put them at the top. They’re at the bottom with the fast tests. And now the flow test. Some of them are medium because we’re only going to test the flow of something like request $20 and it receives $20, and we get back an indication that we’re going to get $20. That’s a small little piece of the entire flow, cause we need to log on to our ATM, then get the money out and so forth. So we can have some of these ATDD given-when-then being medium. And then some of them are going to be the large, cause I do want to run through the entire scenario of sticking my card in, putting my PIN in, looking at the silly advertisement they’ve decided to put on the ATM and saying, no, I really want to just get my money, and then entering my amount, receiving my money and my card. It’s a long test, but you got to do it. But we only want one of those. So it’s an end-to-end test. Well, okay. So at that level, it’s the system test, cause it’s testing the entire system, yeah. So ATDD falls on that level, also falls in the unit level, could fall on the integration level if you’re doing a small little given-when-then. So where does it fall? Everywhere.

Now, you mentioned API. You’re going to come up with an acceptance test for your API. And you’re going to come up with tests for the individual methods or the individual calls. And then you’ll probably have a few tests that involved multiple calls that you want to say, okay, I called this, and I called that and I called that. Did everything work out right? Those are acceptance tests. Typically, for an API, you don’t need a customer. However, if it’s a public API, you’ve got customers. Are you talking to them? Are you finding out what they want? Are you finding out how they want their errors or anything to be returned? How you want error handling and so forth? So it works on a smaller level. Somebody is the customer for an API. Somebody is the developer of it. And you might have a tester come through once again, they could go, “Okay. So what happens if I call this, but, oh, this isn’t working? Will this at least return me the right thing.” So you’ve got a smaller thing, and it’s an internal acceptance test for an API. It’s not an external giving somebody money out of the ATM.

[00:33:22] UAT & Reporting

Henry Suryawirawan: Thank you for clarifying this using test pyramid. So if you can classify your tests into like small, medium, and large, probably you can easily deduce which test should it be. And you mentioned about acceptance test should be everywhere. Maybe one kind of a confusion, maybe also personally myself, right? Sometimes we treat acceptance tests as something to be reported. So think of it like a user acceptance test, right? Before you can deploy your product or your feature, you need to have this report. And if you mention that this acceptance test can be anywhere, how do you report it easily? So that maybe is my question.

Kenneth Pugh: Okay. So let me take this part of your question, and then I’ll get to the reporting. User acceptance test. Okay. That’s something you give the user at the end and they go, okay, is it working or not? How come you didn’t talk to that user ahead of time and say, “Look, when I give you this application, what are you going to do? Can you just tell me the steps you’re going to follow?” Oh, okay. Well, first of all, that might change what you actually implement. But secondly, why aren’t you running that? Why are you waiting for the user to run it? We find those users. We ask them how they’re going to be using this system. They’re the ones who should be helping create the test in the beginning. I will make my exception. So, usability test. Okay. Because until they finally see what the GUI looks like, and they’re actually using it, well, you can do some prototyping. But in reality, they’ve got to have a hands on. “No, this is too many clicks”. Cause you can only do so much with prototyping. And so there still is that usability, but the functionality should’ve already been taken care of.

Now as far as report goes. What are we reporting? And why are we reporting? If I have acceptance tests for every requirement and every requirement is required, then my only report is everything passed. And I shouldn’t have, oh, we got a couple of failures here. Well, why do I need to report it? We have failures, fix them. So your only decision is everything works or these few things don’t work and we’re going to fix them, which is a no brainer, or these things don’t work, and we’re going to put a feature flag or something so people don’t see them so we can ship the system now, or these things don’t work and we just decided we’re just going to take these out. So like we can remove the test cause we remove the requirement and we’ve gotten rid of the failure by removing the requirement. So those are your decisions, but basically, the entire application should work as is. So it’s a 100% acceptance test.

Henry Suryawirawan: So I think, yeah, go back to the CI/CD pipelines that you have. The number of tests that you have there. So if anything breaks there, even though it’s unit, integration, whatever you want to call and classify it, if something breaks, then yeah, you have the software not working based on the specification.

[00:36:21] Automating Acceptance Test

Henry Suryawirawan: One area, of course, about acceptance tests, people sometimes heavily associated with the automation part of it. Writing automated acceptance test or many people call it BDD. So maybe we can go there about the automation part of the acceptance test or this given-when-then kind of thing that you must have seen in the industry these days. So how can people take acceptance test maybe defined by the triad and then automate that?

Kenneth Pugh: So in the old days, and I’m talking about early 2000. At the beginning of the millennium. There was an acceptance test framework called FIT, Framework for Integrated Testing. And it worked pretty well. It was a table based system, and it was perfect for doing business rules and things like that, and it had automation, then it’s now sort of turned into Fitnesse, which is FIT with a Wiki. For I’ll call it the modern day, you’ve got the variations of Cucumber. There are other automation frameworks, but I’ll call it the Gherkin. The Gherkin style is the one that I tend to use. It’s simple to write. If you are consistent, it’s simple to automate, and just a little discipline in how you write this stuff. So, if I have a given-when-then, and most of my given-when-thens have some data associated with them. I’ve got a bank account that has a balance. I’m taking so much money out. I’m taking $10 out and therefore, my balance was $20. Now my balance should be $10 and so forth. You can then automate those. You can turn them into automation in approximately 30 seconds to a minute. So we can convert that given-when-then into a code in whatever language you like, Java, C#, which uses their version is called SpecFlow instead of Cucumber or Ruby, Perl, Python. You name it. So now you’ve got the template that gets the data from the Gherkin. And now, you got to write some production code. If you want multiple asserts in the then statement. But then you got to write some production code. Here’s my data. Here’s what I’m expecting from my output. Guys, let’s start coding cause we got some work to do.

So the automation part is trivial. In fact, both in Cucumber and SpecFlow, it’s basically you have a feature file that has the Gherkin in it. You push a button and it will actually create a skeleton code for you. It even has put your implementation here, put your call to your implementation here, and so forth. So it’s pretty nifty and, as I say, very little overhead.

[00:39:09] Table-Driven vs Text Format

Henry Suryawirawan: You mentioned something that also picks my interest. So you mentioned about table-driven specifications versus textual format kind of specifications. I saw that maybe through your blog posts and things like that, that you promote a lot about table-driven test format. So tell us more why probably table-driven is more beneficial or maybe more useful than the textual simple format?

Kenneth Pugh: So this gets into how much data you’re having to deal with. In a standard Gherkin, you might say, given my balance is $10, and I have had three withdrawals, something like that. That’s nice. For simple stuff, it works perfectly fine. But now, when you’re starting to get more complicated, given my account is, and now I want to have a table of the attributes for that account. Is that account an object? I don’t care yet. We’re not dealing with that, but it’s an entity, and it’s a domain term that everybody would understand. And the attributes would be something that product owners and the SMEs would understand.

So now we have balance. In our header, we have the number of withdrawals in the past week, and so forth. Each of those data items becomes a column header, and then we give it just the data underneath. Well, what that does is it draws out the attributes, those domain terms for each of those fields, better than if you just sort of string it in a sentence. Because it’s like, okay. The balance is and it’s like, “Uh, the term is what?” Oh, balance. Why don’t we just put that as a header? If we put that as a header, it’s already parsed out for you and everything. So having these step tables, if you will, the data underneath each step in a table gives you an easy way to look at the domain terms, an easy way to look at the values each.

I always say it’s in the language of business. And everybody looks at me and says, “What are you talking about?” I said, “Every business person uses Excel. They’re all used to tables.” So having this in a tabular format, it’s like, yep. That’s easier to read than these 50 lines of here’s some input and everything. So that’s why I emphasize the tables. It also turns out there’s some other interesting, easy things. It’s very easy to add another column to a table. And it turns out you don’t have to pass all the values in a table. You can default them in the step definition code itself. So it’s both readable and can be more efficient.

Henry Suryawirawan: Yeah. So one counter argument that sometimes I observe by business people looking at the Gherkin. Although it’s in plain, simple language, natural language, sometimes, yeah, it gets too maybe granular in such a way that is very difficult to grasp what this test is all about just by looking it simply. So you mentioned that business people tend to use Excel spreadsheet in order to define their business rules or maybe some examples of how this should be done. I think, yeah, that’s correct. Actually, by having tables, you focus the attention into the headers, which are probably the attributes and the domain terms, the important concepts and the variations of values that could potentially happen and what your code should assert in terms of a correctness. So thanks for emphasizing that. I think a table-driven, I’ve used it in the past as well for insurance calculation. That actually is the main language that the actuarial people use to actually define the formula for this calculation.

[00:42:46] ATDD

Henry Suryawirawan: So Ken, we are reaching towards the end, but before we go into the last question, so I want you to probably give us some advice for people who are building products or building software. What should they do in order to kick start this journey of acceptance test-driven development? So it’s not like acceptance test last.

Kenneth Pugh: What should they do to kick start it? Well, I would take a class, and I want the team to adopt it. What happens sometimes is when individual goes off into class and comes back with this great idea and it’s like, “Oh yeah. Okay. That sounds like a lot of work.” And by having a team take the class because it’s a team that needs to collaborate together. They need to have a common understanding of how everything goes. So that’s why I teach my classes to teams, not to individuals. Having the team get together, do it on their own stories and realize how much they may have learned from actually creating these tests, and how much the clarification of the requirements come by doing this first, which is, of course, the developer’s always biggest complaint. “Oh, the requirements are unclear. Ah, test is absolutely, you know, it’s yes or no. Either your pass it or you don’t.” And so that’s how I would start. I mean, you can try it on your own.

I have workshops that I teach for teams. We basically go through and we use your stories as the exercises, not my ATM example. We use your stories and you come out of the workshop with some of your stories that are fully developed. And now you may be able to recognize here’s the difference between what we were doing and what we could be doing.

Henry Suryawirawan: That is always nice to compare or contrast between what you were doing before and after the new knowledge, how you should do it. It gets started from there and hopefully the behavior and the habit stay on and you have a much more improved software development life cycle.

[00:44:49] 3 Tech Lead Wisdom

Henry Suryawirawan: So Ken, it’s been a pleasure talking to you. Unfortunately, we reach the end of our conversation. But before I let you go, normally I would have one last question that I always ask for all my guests, which is to share about three technical leadership wisdom. So think of it like advice or maybe some kind of insights that you have retrieved from your journey so far in your career. So what would be your three technical leadership wisdom?

Kenneth Pugh: Okay. Technical leadership. A. Trust your people. You hire good people. Let them make the decisions and let them do their job. That would be number one.

Number two, encourage consistency so that people can move from a team to team and have a similar process or a similar environment in which to work. But don’t demand it. Encourage it because there will always be cases and exceptions to any standard you come up with. So go, “Here’s what we really encourage you to do, but we’re reasonable for exceptions.”

And then three, and it sort of goes along with the you’ve got the best people. When you’re making decisions, involve them in the decision. Get feedback. Now, there are always times where it might be. “Oh, okay. We got to do this cause we got a half an hour and we don’t have time.” But if you’re making changes, let there be time for the change to, I’m going to call it, be accepted before the change is made. So you’re going to clarify why we’re doing the change. You’re going to ask for is this the right change? And what are the pitfalls of the change? So instead of going downwards, ask upwards, and try and form a cohesion naturally. Now there will always be the tail ends of the Gaussian curve, and you will never be able to satisfy everybody. But let’s at least move the mean over to accepting the change that you wish to make rather than just going, okay, next week we’re going to be doing this.

Henry Suryawirawan: If I may try to relate it with acceptance test as well. Before you actually make the software, the decision, make sure that you involve the people to try it, to actually chip in and give comments and maybe agree or disagree with all those before you’re actually down to implementing it.

So Ken, it’s been a pleasure. Thank you so much for sharing everything about acceptance tests. I learned so much today. For people who would like to continue this conversation or maybe get into your class or resources, is there a place where they can find you online?

Kenneth Pugh: Yeah. You can find me in LinkedIn, Ken Pugh, and I am Ken Pugh cause I was the first Ken Pugh on LinkedIn. Or you can go to KenPugh.com, KenPugh.com. And you’ve got all my information there.

Henry Suryawirawan: Any books that you’re going to write lately?

Kenneth Pugh: Well, actually, I am thinking of writing two books and they’re sort of parallel books. One is on effective software development, which is going to be a book on doing agile or even non agile, and going with patterns on how to develop software and picking the right patterns in your development process. Some of them, if you pick one set of patterns, it would look somewhat like Scrum. Another, you might look like SAFe. Another by Kanban. So it will be a sort of a pattern selection book.

And then there’s another one. I’ve got sort of like just a little basis for it. And that’s basically a coalesced view of software development. It takes in a value stream and acceptance testing and how to break things into scenarios that become stories and so forth and so on. And it’s sort of like a one way. I always call it one way cause there’s always three ways to do anything. The best way is whatever works in your context. And so this would be a book on here’s one way to do things of all the possible combinations.

Henry Suryawirawan: Looking forward to seeing those books published. And if you have it published, I guess I can have you back in this podcast.

Kenneth Pugh: Okay. Sounds good.

Henry Suryawirawan: So thanks again, Ken. Thanks for your sharing today.

Kenneth Pugh: You’re welcome. Take care.

– End –