#148 - Behavior-Driven Development (BDD) Essentials - Jan Molak & John Ferguson Smart

[00:01:30] Episode Introduction

Henry Suryawirawan: Hello again to all of you, my listeners. Welcome back to the Tech Lead Journal podcast, the podcast where you can learn about technical leadership and excellence from my conversations with great thought leaders in the tech industry. If this is your first time listening, please subscribe on your favorite podcast app. You can also subscribe to Tech Lead Journal contents on various social media on LinkedIn, Twitter, and Instagram. And for video contents, subscribe on YouTube and TikTok. And if you have been enjoying Tech Lead Journal, please support my work by either buying me a coffee at techleadjournal.dev/tip or becoming a patron at techleadjournal.dev/patron.

My guests for today’s episode are Jan Molak and John Ferguson Smart. They are the co-authors of “BDD in Action: Second Edition”. In this episode, we discussed in-depth behavior-driven development (BDD) and its essentials. Jan and John first began by introducing what BDD is, the benefits of using BDD, and the Gherkin language with its given-when-then syntax. They gave advice on how to introduce and apply BDD, especially for legacy software, and how to manage the BDD specifications effectively.

Jan and John then shared several BDD techniques, such as feature mapping, example mapping, impact mapping; and went deep into the screenplay pattern and the Serenity projects they both create to implement the screenplay pattern. Towards the end, Jan and John share the insights on which testing layers we should apply BDD and some anti-patterns we should avoid.

I hope you enjoy listening to this episode and learn a lot about BDD and good practices for managing user requirements and automated testing. And if you know someone who will also benefit from this episode, please help share it with your colleagues, your friends, and your communities, and leave a five-star rating and review on Apple Podcasts and Spotify. It will help me a lot in getting more people discover and listen to this podcast. And I really appreciate it! Let’s go to my conversation with Jan and John after quick words from our sponsor.

[00:04:00] Introduction

Henry Suryawirawan: Hello, everyone. Welcome back to another exciting new episode of the Tech Lead Journal podcast. Today, I have with me two guests in one time, right? So they are the co-authors of a book titled “BDD In Action”. So if you are into software testing and if you have heard about Behavior Driven Development, today, we’ll be discussing a lot about it.

And this BDD in Action is actually a second edition, in extension to the first edition, which was published I think sometime in 2014. So John Smart was the first author of the book, and then in the second edition he collaborated with Jan Molak here to come up with the second edition. So, really looking forward to this conversation and learn a lot about BDD and software testing in general. So thanks for being here, guys.

John Ferguson Smart: It’s a pleasure. Thank you for inviting.

Jan Molak: Thank you for having us.

[00:04:49] Career Journey

Henry Suryawirawan: So Jan and John, I always like to ask my guests to introduce themselves a little bit, maybe telling about career highlights or any turning points. So maybe I’ll leave it to either one of you to start first.

John Ferguson Smart: Well, I can go first. So I’m John Ferguson Smart. I’ve been in software development for more years than I can remember. Since the 90s. So I’ve been doing agile software development since the late 90s, early 2000. I’ve done a bit of everything. So development, I still do active development, testing, project management, waterfall, agile, XP, the whole, whole range of things. In the 2000s, my role was very technical and I always thought the requirements side of things was a little bit intimidating. All this talking to people and stuff, I thought that was a little bit of a challenge.

So I gave myself the challenge of actually getting into that and doing that a little bit more. And that’s where a lot of my activity in the sort of BDD space emerge. I started to get involved with people in the sort of agile and BDD communities in London quite a bit. The Dan North or Liz Keogh, people like that who were very active. Chris Matts was another one, very active in that area. And so that’s where my big interest in Behavior Driven Development really came from.

But if you’re talking about highlights, actually one of the interesting highlights was, I’ve actually been doing BDD for well over 20 years, because, one, there was this one time I was working with an insurance company in Paris, France. And their problem was they wanted to put an insurance application online and their problem was reproducing their mainframe insurance algorithm in Java. And they had like a gazillion, lots of rules and were all done in this really complicated Excel spreadsheet. So, I had the idea of taking that Excel spreadsheet and actually saying, well, give me all the examples that you use to test your Excel spreadsheet. And we used that to test the actual code we write.

And that was effectively, we just ran it through JUnit and produced a little report and there was actually another Excel spreadsheet. Some words were green, some words were red. And that was the first executable specification with collaboration with the business that they ever wrote. And I thought that was such a wonderful technique that works really, really well. I’ve been sort trying to do it ever since. Cause I really do think it is by far the most effective way of collaborating and delivering software that makes a difference that I’ve experienced.

Henry Suryawirawan: Wow, that must be a very tremendous effort. So I really congratulate you to have a success for that project. How about you, Jan?

Jan Molak: Oh, so my career journey actually didn’t start in finance or banking where I work right now. It started in a complete opposite of the spectrum. So it started in early 2000s in the video games industry, when I was back in Poland. So, I remember one of my first jobs, I was actually working for CD Projekt Red, the studio behind the games like the Witcher, for example. I was the very first full-time web developer at there, and my job spec had one sentence, said, well, sort out everything around the internet.

First of all, well, that’s quite broad. Well, I asked my boss, well, what exactly do you need? And he told me, well, what do you think we need? So I said, well, we would probably need to have a place for people to exchange comments about the game. We need to have a place to publish the news about the game. I said all those things and he was like, well, that’s awesome. Let’s do it. How can I help you? So remember, that was my first job. I was like, you know, in my early twenties. I had absolutely no idea what I was doing.

But what it really helped me to do is to actually have this business perspective on software. So how can I, you know, create software, how can I work with my team to create software to support a, back then, small company? So I then use now, the things I learned back there and tried to apply it to other industries. Now, later on, I moved from Poland to the United Kingdom. And that was similar to John. That’s where I met so many people who are now practitioners of eXtreme Programming, of Test Driven Development, of BDD and so on. And one of the problems I was trying to solve, and I was trying to find the answers was, well, how do we automate testing?

Now, not sure if the audience knows about it, but one of the main problems in, or one of the main challenges in the video games development is how do we actually test the games? If you have a like a real time simulation, cause that’s really what a video game is, well, how do you make sure it works? So I tried to find answers to that and you know, typically the answer I would get is, well, it’s not possible. No, it’s just too hard. And whenever I hear it’s not possible, it’s just too hard, the first thing I do is I try to figure out why it’s not possible. Then the second thing I do is try to figure out how to do it.

So I try to actually ask this question now across different industries. So I worked in publishing, I worked in media, I worked in broadcasting, I worked in finance. And one thing I actually realized is that now, whatever company I joined or whatever client I worked with is how very often I hear that testing is impossible, it’s very hard to do. Well, why is that? So many of those companies, what they would do is they would, you know, hire manual testers to click through the scenarios, and basically, confirm, you know, whatever developers have already created.

Now what I realized is actually, you know, testing is difficult because it’s actually very hard to understand what is it that we are actually trying to test, why you’re trying to test, how to set the priorities, how to understand what parts of the system are actually important. And turned out that just in video games, there was the same case in finance, in insurance, you know, in banking, in broadcasting, everyone was the same case. So then I started looking at two ways section, well, first, automate my tests, then figure out how to gather the requirements, actually how to understand what bits are important. And that’s how I came across John’s work around Behavior Development. That’s how I got to meet guys like Dan North and Liz Keogh and the rest of the London BDD and TDD crowd.

So that’s how, you know, I actually got to connect all those different dots and actually figure out that in order to avoid issues with manual testing, in order to automate testing, well, you actually need to understand what is it that you’re trying to test, why you’re trying to test it. Now, to do that, you need to understand your requirements. To understand your requirements, you need to speak with the business stakeholders, you need to speak with your customers. You need to have ways to actually gather and analyze feedback in very fast and efficient ways. So that’s how all those different pieces of my career journey came together here in London.

Henry Suryawirawan: Thanks for sharing your story. I think throughout both of your career journey and highlights, I can tell that you guys deal a lot with software requirements and testing, right? And the challenges, how BDD later on can help in terms of doing software testing and also gathering requirements.

[00:11:30] Challenges Before BDD

Henry Suryawirawan: So let’s just go straight into BDD, right? So in the first place, maybe if you can summarize, what were the challenges before BDD was around, right? What were the challenges that probably was prominent in the software development world?

John Ferguson Smart: So from my perspective, BDD, I’ve heard it described by one of the managers I’ve worked with as it’s what makes you actually be agile, not just do Agile. And so, BDD and Agile are very closely related. That is all about solving that same problem. It all comes down to the communication. So we can code all we want, but coding is not really the aim of the game. The aim of the game is to deliver solutions that actually solve problems that the business have. Enable them to do something that they couldn’t do previously. Enable them to do something that they could do previously, but better. Prevent problems, save money. Where there are the four categories that we know about how you can actually deliver value, Chris Matts' four categories, if I recall correctly, how you can actually deliver value and all of that.

The problem is it comes from understanding the requirements. And what we’ve always found is that you can’t just write down the requirements or have someone write down the requirements for you and expect to get value out of them. There’s always iterations, there’s always feedback cycles. And the aim of the game is to actually reduce those feedback cycles and the waste that come from that.

And before, well, before the days of BDD, you would inevitably have really long cycles where you’d have siloed processes. And even in allegedly agile projects, you still get that siloed processes of BAs or analysts or product owners write requirements hand them off to the development team to build, they build something. They hand that off to testers to test, and then they, the testers, or maybe they even also have a UAT testing phase, where users go through and test things in their own way. And that’s always a bit of a surprise cuz they never told you what they were going to actually test. All of that, every stage gives you the opportunity to lose information, to make mistakes. And so that’s what the biggest challenges we were having before BDD is getting that communication aligned.

Jan Molak: Yeah, I think that’s actually quite a funny thing. Cause now typically, when we think about software development, we focus on things like the languages we use or the frameworks we use or the automation tools we use. But we kind of forget about the most important bit, which is that the social aspect of creating software. So, BDD is one of those tools. So Behavior Driven Development is one of those tools that helps you to focus on the actual outcome that your software or your solution is supposed to bring to the organization. Now, the interesting thing about it is that bringing the outcome doesn’t necessarily need to require software. So that’s again, one of the things that software developers very often forget about.

I mean, if you think about it, you know, I’m a software developer, therefore I have to build software in order to exist, right? I mean, that’s my job. That’s my role in this system. Now with BDD, what we can do is we can look at the outcomes that our organization seeks to achieve, and then perhaps we could try and find solutions that don’t require us to build any software. You know, perhaps we can bring the same outcome by improving the process by speaking to another person. By reusing the tools we already have. Now, if we can accomplish the same outcome without building additional software, well, then we won’t have any additional software to maintain and therefore we can further reduce the costs.

Henry Suryawirawan: Thanks for the addition to that important bit, right? So we don’t always have to need software to bring an outcome for the business or the organization. It could come in many different shapes and forms, right? It could be process improvements, maybe some kind of prototypes, maybe some kind of a cheaper solution than writing software.

[00:15:24] Behavior Driven Development (BDD)

Henry Suryawirawan: So let’s go to probably the definition of BDD right? So for, people who are new to BDD, also heard people sometimes mistaken BDD with just the Gherkin, right? The given-when-then. Some people also refer to ATDD (Acceptance Test Driven Development). There’s also people are saying Specification by Example. There are so many different terms. Maybe if you can help to define first, what is BDD, and how does it compare with all these different terms that are available out there in the industry?

John Ferguson Smart: So Behavior Driven Development is a collaboration approach. From my perspective, it’s not a particular tool set or not a particular… It is a methodology, but it’s not related to a particular language or tool set or testing approach. It’s definitely not about test automation. And it does encompass all of those other aspects nowadays. I mean, it’s a concept that’s been developed in different ways. Back in the noughties, we had FitNesse. We’d call that ATDD because we’d try and define the acceptance tests first, just like we’re doing TDD, and we’d try and write them in a business readable format. Specification by Example was Gojko Adzic’s book, which described that same approach. Behavior Driven Development was originally coined by Dan North as a way to explain Test Driven Development. But with the work of Chris Matts and Liz Keogh and other people in that community, it fairly rapidly expanded into being applied to analysis. So often we see BDD extends from TDD. Now, it does not. BDD is a totally much broader concept now. It’s a collaboration process where we involve business users, developers, testers, everyone to have conversations around the requirements. In BDD, one of the concrete things that we use, one of the really characteristic aspects of BDD is that we use examples a lot. So rather than just writing general specifications, we describe everything in terms of examples. And we use those examples to drill into the details and challenge the assumptions and challenge our mental model of what a requirement actually is about. So that using examples and counter examples and telling concrete stories is a much better way of getting the team involved and engaged than the more traditional requirements writing approaches and even just classic user stories, which can very quickly become just a proxy for requirements.

Now, the automation aspect, BDD is not about automation. Cucumber is not an automation tool or test automation tool. We’re not in the business of automating test cases. What we’re trying to do is define the requirements in a format that can give us fast feedback on whether they’ve been implemented. Now, it turns out that a really effective way of getting that fast feedback is to automate them. So automate the requirements using tools like Cucumber. Now tomorrow, if we get a tool that’s better than Cucumber, we can use a different tool. We’ll still be doing BDD. It’s not related to a particular tool.

So Gherkin is the given-when-then language that’s used in tools like Cucumber and SpecFlow and Behave, and quite a few others. And it’s one of the more common ways of doing BDD, but it’s not the only way. We can also do BDD, express our requirements and write them very well just in plain JavaScript or Java or Python. And if we write our scenarios or examples in a business readable language, and if we can produce reports that reflect that business readable language, then that’s BDD as well. So I’ll tell you that in a nutshell is sort of what BDD is all about. I know it is a very broad definition, but there is a lot to it.

Jan Molak: Big one thing I would like to add is that what BDD helps you with is helping to avoid things like scope creep. So focusing your team on outcomes. So focusing on the conversation around outcomes. So what is that our software system, what is that our solution is supposed to bring to the organization? What it does, does several interesting things. So firstly, we talk a lot about our collaboration and conversations and so on. What many people might hear, well, oh my God, we’re just going to have more meetings. No, no, not at all. What we mean is that what we are doing with BDD is we are helping team to start having informed conversations or focused conversations around specific outcomes they are trying to accomplish. Now, it’s just not any random chitchat. So we are trying to focus on the outcomes, because focusing on the outcomes helps us find more precise solutions to the problems we are trying to solve. We are not just shipping features for the sake of shipping features. We want to solve a specific problem.

Now, this focus also helps us to avoid scope creep, as I mentioned earlier. Now, the idea here is that, you know, if you know, that your goal is to, I don’t know, increase the number of conversions on your homepage by whatever, 20%, by some time, well, then you are trying to focus on delivering those features that help you to accomplish this specific goal. Now, this helps you to avoid having to ship any other features that don’t support this goal. Now, what this focus also gives you is also gives you an opportunity to improve the overall quality of your software. So again, not only we can turn those requirements, those scalable specifications into automated tests by reducing the scope and focusing only on those things that matter, it’s easier for us now to ensure the quality of the software we deliver, simply because we have less of it to deliver in the first place. And now what we deliver is of higher value. So again, we are focusing on delivering high value software, and making sure that it’s actually the software itself is of high quality as well, not just churning more and more features.

Now, there are many other interesting outcomes of this approach. Now by, again, focusing scope, improving quality, improving the value of software we develop and deliver, we also improve things like developer happiness. Because now suddenly, now that, you know, have conversations with your business sponsor, if you have conversations with your customers, you understand what is that you’re trying to solve now, why you’re trying to solve it. You actually have a mental image of the person you’re actually delivering the solution to. It’s much easier to now see how your work is worthwhile. It’s easier to see now why you’re actually trying to do it. Now, this, again, you know, improves developer morale, improves developer happiness, increases developer productivity and so on. So it’s actually quite interesting how many positive effects a team can see by doing things like focusing your work on the outcomes that you are supposed to bring to your organization.

Henry Suryawirawan: Thanks for highlighting the important aspects of BDD. I think the first thing I heard very clearly from John is that BDD is not automation only, right? It’s not software testing only. The first is about collaborative approach that focuses a lot on conversations, examples, right? And also outcomes like Jan also extended in the later part. So I really love that explanation. So for people who still think BDD is just another tools for automation to make your software testing much better, I think it is more than that.

[00:22:33] Benefits of BDD

Henry Suryawirawan: Which speaks to the next topic that I’d like to ask you guys, what do you think are some of the benefits, if you can summarize, that BDD offers for people to give it a try? If they don’t have the time yet to try BDD, why they should consider applying BDD now in their software development practices?

John Ferguson Smart: From my perspective, and this is what I see on the ground, it’s a good question because you do need to know why you’re introducing any technique and what you are going to measure it on, and whether it gives you the outcomes, cause that will tell you whether you’re actually implementing it effectively or not. And there are two levels of outcomes that I see. There’s what you might call the proxy outcomes, which are easier to spot, easier to see straight away. And then there are the longer term outcomes, which was what you’re really trying to achieve.

The sort of proxy or short term outcomes, because of the nature or the collaborative nature of BDD and the fact that you really do build up a shared understanding within the team of what you’re trying to do, it tends to reduce the number of defects dramatically. So I see, typically, and this is not talking about automation, just the fact that you collaborate effectively reduces a typical defect rate by 70, 80, 90%. You just don’t get defects into production a lot of times, because you get a much deeper understanding and you’ve got the opportunity to ask questions and people do ask questions.

Now, that assumes that you are actually doing the collaboration correctly. If you are just writing given-when-then statements in your test cases and calling it BDD, you won’t get those benefits. And you’ll wonder why it doesn’t work. But if you do the collaboration and then think about how to automate, you will get those benefits. The automation, BDD is a really effective way of doing automation. So I find that when you do it well and combine it with TDD, you can do a really nuanced level of automation and get a lot of bang for your buck for the amount of automation effort that you do. So it really helps you get a laser focus on what you should automate, why you should automate it, and where you should automate. And so that’s a good practical benefit I find.

There’s also a longer term benefit is that because you’re having these conversations, and this applies, you want to be having conversations not just at the story level, but also at your features and epics and in coming up with examples of user journeys and use cases at the epic level, so you can understand what they’re about and where the value is coming from. And that helps you to be more confident in delivering value that actually delivering things that make a difference to make an impact. So that’s a really important longer term benefit.

And the third big benefit that I see, and which is a little bit less tangible, but is what a lot of people report to me and what I see in the teams that I work with is that you get the teams which are much more engaged. So team members tend to be much more engaged and creative, because they know what they’re contributing to and they have a sense of mission and a sense of ownership of the problems they’re solving. Much more so than a traditional siloed approach where often development activity is basically implement this JIRA without really understanding the bigger picture very much. So the teams do get much more engaged and that leads to sort of better productivity and more creative outcomes.

Jan Molak: Yeah, I think one of the things that John mentioned, they really deserve to be highlighted even more. I mean, if you think about it, according to various scientific research done over the last couple of decades, now, it turns out that between, I think 40 to 80% of all software defects actually come down to issues with requirements. Issues with misunderstood or perhaps miscommunicated requirements. Now you asked about the benefits, Henry, so if, you know, in our audience, we have listeners who are on teams that suffer from issues with production defects, issues with defects in their software.

If there was one thing for you to improve, think about what can you improve around your requirements, around how your development team collaborates with the business, with test team if you have one, with ops teams and so on. Think about improving how you collaborate around the requirements, because issues with requirements cause most issues with your software most probably. So by addressing issues with requirements, you are more likely to improve the quality of your software than by focusing issues with, let’s say, coding errors, which are much less frequent than issues with requirements.

Henry Suryawirawan: Well, thanks for the plug. I think remember this kind of research as well, and I also know that rework is also something due to the miscommunication or maybe the requirements not captured properly. So I think, yeah, if software engineering team wants to improves some parts of their productivity and effectiveness, probably requirements is one big area where it’ll bring a lot of impact. So thank you for expanding some of these benefits. I really love that John mentioned about this increased engagement from not just engineers, probably, it’s the whole team, right? Because they’re so collaborative and trying to come up with a better way of approaching the problem and achieving the target outcomes.

[00:27:39] Gherkin

Henry Suryawirawan: So talking about BDD, I think we talked about it earlier just now about Gherkin, right? I think talking about BDD, we have to cover a little bit about Gherkin. So what is this Gherkin, why the name is so funny, and what is the essence of Gherkin anyway?

John Ferguson Smart: Gherkin is a small cucumber, pickled cucumber, kind of. It’s in that family of vegetables. And so there’s a relationship between Cucumber the tool and Gherkin the language. So Gherkin is the given-when-then notation that we use. So Dan North was originally at the origin of that, where coming up with that notation. And then Chris Matts observed that it was a good way of describing business expectations as well. But a lot of people misinterpret it. But a lot of people, they don’t know how to use it, basically use it incorrectly.

And it’s what happens when we see test scripts written in this given-when-then format. Given-when, that format, it can be crystal clear. It can be a really nice, precise, readable way of writing your requirements. Or it can be a horrible mess. It’s like anything. It can be, depending on whether it’s well written or not so well written. It will have value or not. So there is quite a skill to writing a clean, effective, concise given-when-then statement that really reflects a business rule and really reflects a business requirements. And so it does take some practice and some effort to get that right to start with. But when you do, what happens is you end up with this, what we call a domain specific language, which is really the heart of the collaboration for a team around a set of requirements.

And that’s where I’ve had projects where we’ve defined this sort of domain specific language for a particular part of a project, and it’s allowed business to write their business rules and express their business rules with such clarity that there’s no other automation that needs to be done. They just need to express a new business rule using their existing language, and everyone will understand what it needs to do and what needs to be done. That doesn’t happen in all projects or in all cases and with all situations, but when you aim to do that, you can come up with that language and use it as a really central part of your requirements definition process.

If you just use it for test scripting, it’s much less valuable because a test script with given-when-then, generally, people will mix up the given-when-then, and it won’t make any sense. It’ll be hard to read and also when people write test scripts with them, they tend to be much more granular, much more focused on clicking on buttons or clicking on fields. And that becomes brittle and hard to maintain. And so people say Gherkin’s no good, because it creates test scripts that are hard to maintain. Absolutely, it does. But if you use it to write executable specifications in the business language, it can be extremely effective.

Jan Molak: Absolutely. One of the main things that, you know, the Gherkin language is supposed to help you accomplish or what aims to help you accomplish is a separation of preconditions, actions, and outcomes. So given-when-then, it’s actually very similar to what you’d see in well-written unit tests with the separation of three parts, so Arrange, Act, Assert. Now, we are doing something very similar, but with the focus on business requirements. Now, the reason why we’re expecting this in a human readable language is so that now we can create an opportunity to avoid some implementation specific details. So again, we can focus on the business language. We can focus on introducing this business domain language into our executable specifications.

Now, an interesting thing about Cucumber and Gherkin that, you know, many, especially English native speakers forget about, is that what Cucumber and Gherkin allow you to do is they allow you to express those executable specifications in a language other than English. Now, Cucumber, the last time I checked, it supported over 70 different languages. So what you can do now as a team, if your business sponsor or your audience speaks language other than English, or they’re more comfortable with other language other than English, then you can express your specification in Spanish, French, Arabic, or whatever other language you prefer. And then Cucumber under the hood will translate, you know, those statements in the human readable language to your actual automation calls. Now, this is gonna be very, very useful, again, as a tool to improve collaboration with different audiences of your automated specifications.

Henry Suryawirawan: I wasn’t aware that many languages are supported. So thanks for adding that. So I think that’s really interesting.

Jan Molak: Oh, yeah. Even Pirate Speak, from what I remember. So those arr.

Henry Suryawirawan: Right. And also I just knew from John what gherkin means, right? It’s a small cucumber. So I think I learned these two things really well today from you guys.

[00:32:24] Introducing BDD

Henry Suryawirawan: So talking about BDD, right? I think in order to introduce that, I think it’s not just a simple switch, right? People need to learn and people need to understand how should the flow be. And I think, I believe many teams these days practice agile software development methodology. Whether good or bad is a separate discussion. But I think in your book you outline several steps that people can try in order to incorporate BDD in their software development flow. Maybe if you can give us a highlight and outline how should people introduce BDD seamlessly in their software development methodology.

John Ferguson Smart: So we work with teams, Jan and myself, that’s what we do, we work with teams to introduce BDD into the… So we’ve got a fair bit of experience with what works and what doesn’t. And what we’ve always found is that, you need to start with the requirements part. You need to start with the conversation. The automation comes later, but you’ve got to start with the conversation. So you’ve gotta get people communicating and collaborating and having those conversations. And it might feel like, yeah, I was saying earlier, it might feel like you’re spending more time in meetings. But actually, what you get out of those meetings is multiplied many times by the time you save, actually delivering the code that you’re meant building and delivering the code and the features that add the value.

So, the first step is to understand what the BDD process is, understand that it’s about conversations and collaboration and then automation, and start structuring that, uh, those making time, carving out a time in your existing process to have those conversations. You don’t wanna leave it ad hoc and you don’t wanna leave it to sort of just random conversations.

So the second thing is that BDD proposes a number of practices, like example mapping, feature mapping. There are a whole lot of others, but they’re structured practices which are designed to help teams get the most out of those conversations. And they are very intended. There’s a whole lot of psychology that goes into a lot of the things we suggest in BDD. But those techniques are really important, especially when we have teams where some people might be more vocal, some people might be more reflective and like to take time to get their answers. When we can use collaboration tools and when we use tools like Miro and Figma and Mural and whatnot for online collaboration, we can get the same effect. We can use a structured approach to coming up with those examples and then turning them into an executable format.

The third step we introduce is the actual automation side of things. So how do we find an automation approach that will work for the technologies that the teams are using? And that obviously varies and I’m very reluctant to say to a particular team, you must use this automation tool. I’d rather the team own the automation technology. And then we, basically, what we do is we’ll guide them and to choose how they can actually implement. And I mean, we have our preferences, but I’ve worked with teams who work with APL. I remember one team was working on and all sorts of really obscure stuff. So not everyone is using Java or JavaScript. So Java, JavaScript, and Python are the big ones that people are using these days. And with them Cucumber does provide some nice options, but there are lots of options out there and you want something that the teams embrace and adopt themselves.

And then the final stage is actually getting the teams doing the automation themselves. So automation is not just about the tester automating test cases. It’s about we want an outside-in approach. So whereas far much as possible, you don’t automate after the fact. You automate before you actually start the work. And that’s what I was saying earlier on. The ultimate end goal of that is that you don’t need to automate at all. That you express the requirements in your domain language and the automation is already done for you. So that’s sort of the ultimate in ATDD. So they’re typically the stages that we go through that we see when we help teams introduce BDD.

Jan Molak: Yeah, absolutely. I think the collaboration aspect is, again, one of the most important ones here. So, like you mentioned earlier, John, so one of the things that our teams very often try and do when introducing BDD is to focus on the tool. I mean, it’s a natural inclination of any software engineer. We focus on the tool first. So we think, oh, okay, well, I like BDD, I like all the things, the things it promises. I’ll just use Cucumber to write some automated scripts after I’ve already done development. Well, that’s kind of pointless, right? Cause I mean, the development has already happened. The collaboration has already hopefully happened. So now, it’s way too late to actually write any automated tests.

Now, what we do with those workshop that John mentioned earlier is that one of the core practices of Behavior Driven Development is a Three Amigos workshop. Now, the idea with the Three Amigos workshop is to have people representing different perspective involved in software delivery. So the business perspective, the development perspective, and the verification perspective. Someone to actually describe, you know, what and why is needed for the organization. Someone to figure out, well, how we can actually ship it. And someone to critique the idea to find the holes, find the things that might possibly not work, so that we can actually guard against them. So that’s one of the critical workshop that we typically introduce.

Now, another thing that you need to do in these workshops is you need to make sure that people who actually can add value to those workshops are involved. So what a team would need to do. So they need to identify the dependencies, not only in terms of software, but again in terms of other teams. So for example, if I need to accomplish a certain goal that requires me to work with my DBAs or application security teams or QA teams or whatever other teams I have in my organization, I would try to establish those communication links first. So, again, we would typically focus or would recommend the teams to focus on establishing those communication links. Cause this then leads to you understanding the requirements better. This then helps you automate those requirements better. Then this helps you to deliver better quality software.

Henry Suryawirawan: Right. I like the emphasis that you put that we should not start with the automation first, right? Like picking the tools, do the automation, but actually the development is done, and maybe the story was already defined. So I think again, the emphasis is on the collaborative aspect.

[00:38:41] BDD for Legacy Software

Henry Suryawirawan: What about if the teams already have a software in place? There’s a legacy code, there’s a legacy requirements. Maybe written somewhere, maybe it doesn’t get written somewhere. Maybe it’s in people’s head. Is there a way that you would advise people to incorporate BDD into, you know, like legacy software?

John Ferguson Smart: Generally speaking, there are two approaches. Either, you only do it for incrementally for new features, or you backfit executable specifications for your entire application. I don’t generally recommend the second approach, but I have had teams who were really insistent on wanting to do that approach, because they wanted the value of the living documentation. And they knew the legacy application was mission critical, and they had no understanding of what it actually did.

So they, in the case, particular case I’m thinking, which was a company in Sydney who did this and it worked really well, is that they used a BDD approach to retrofit executable specifications for their existing functionality. And that was a way of them understanding what the application did and documenting what the application did. And that allowed them to progress faster afterwards. But in most cases, what I find is a path of lesser resistance is to apply it incrementally and progressively. So you come with a new feature. Your new feature is add a column to a particular search result. You’re not going to write a whole lot of BDD scenarios for everything around it. You’re going to write a BDD scenario for that particular use case and user journey.

So your use case might be, well, an admin will add a product to a catalog with a range of colors and the search screen should show those colors in the search results. And so you might have a sense some scenarios around that, and that will be your BDD scenarios. You won’t explore every possible way the admin can add products of the catalog or every possible way you can search. But as you add new features, you’ll expand on what you’ve done. And gradually, you’ll create these little islands of executable specifications, which will eventually come together and form a bigger picture. And that is a more pragmatic approach when you’ve got an existing application and you don’t want to burden the team too much with having to do too much automation at the start.

Now that said, I do know teams who use that approach, but simultaneously they have a tech debt approach where they recognize that there is a technical debt related to the lack of automation, lack of executable specification. So each sprint, they’ll do a little bit of work to chip away at that and add a little bit more than they need to, so that they can gradually build up their executable specifications more quickly.

Jan Molak: Now, one of the things that could be helpful to teams that actually need to support, you know, existing software while introducing new features is something we described in the book as well. And that’s a technique called journey mapping. So journey mapping is a BDD technique that we’ve developed when working with teams who work on, you know, established software systems. And if you think about it, this a very common case, especially in finance, insurance, healthcare where you know, you very rarely have the luxury of starting with a greenfield project. You typically have something you have to maintain, you need to add to that.

Now, techniques like journey mapping, what they allow you to do is they allow you to focus collaboration with other parts of your organization around the important aspects of the system that your system needs to support. So it allows you to focus on the specific actors or the specific external parties, your system support, the main use cases, the main scenarios, and so on. So what this collaboration technique helps you to do, this helps you to prioritize any of your refactoring or re-architecting work better. Cause you know, now that you understand what the system is supposed to do, you understand what the important parts are, you understand what parts need to keep working whenever you add new software or new features. Then you can make sure, first, your manual testing or exploratory testing activities around those important aspects. Then you can start creating your automation with the focus on the important aspects of the system. Cause, you know, any existing system will have functionalities that are used now on a daily basis or frequently by most of the users. And will have some features that have been added, I don’t know, two decades ago for whatever reason that nobody remembers. So it’s important to be able to distinguish between those two.

Henry Suryawirawan: Right. So start small, start gradually, right? It’s not like one big bang approach. You know, you can have all the requirements captured in one go, right?

Jan Molak: Absolutely. Start small and build up.

[00:43:15] Managing BDD Specifications

Henry Suryawirawan: So both of you mentioned things like feature scenario steps. I think let’s go a little bit more into the techniques, right? For teams who implement BDD, how should they structure all these different given-when-then, different scenarios? Is it common to put it in the user stories or is there a better way of structuring and managing those requirements?

John Ferguson Smart: Generally speaking, some teams will put the given-when-then statements directly in Jira. Some teams will put it in version control. We used to, well, it was very common a while back to actually come up with the given-when-then statements directly and three amigos then automate them later on. But we’ve found that that’s an, a rather inefficient process, because it takes a lot of people, a lot of time to wordsmith those given-when-then scenarios. Whereas what you’re really after is getting the key examples and counter examples. And so the given-when-then really should be what we call an executable specification. And if it’s executable, it’s part of the source code, so it should be actually in the source code.

Now, the problem that happens is product owners and business analysts, they want visibility on those given-when-then. So there are tools around, Jira plugins, for example, that allow you to give you visibility on those given-when-then statements that you’ve got in your source code, even edit them, some of them, so that you can get that visibility from Jira in your user stories. But without compromising the fact that the given-when-then are effectively source code and they need to be in a place where they can be executed. What is an anti-pattern I find is that when product owners write the given-when-then statements directly in the user stories before they have the conversations, because that cut short the whole conversation part of BDD that introduces a whole lot of cognitive biases and actually makes the conversations a lot less efficient. And also they’re really hard to automate, generally.

So that is generally speaking, there’s maybe one team in 10, one team in 20 who can pull that off that I’ve seen. But generally speaking, we don’t advise that. We advise having the conversations first, writing lightweight acceptance criteria first, having the conversations and then putting the given-when-then in version control and then figuring out how to get that visibility in Jira or whatever tool you’re using.

Jan Molak: Plus, I think a very important distinction to make is between features and user stories. I remember now for me to say very important switch in how I was thinking about software delivery as a whole. Features is what your software provides. It’s what the things that people can use. User stories is how you deliver those features. How you structure your software delivery process, you know, in what sequence do you deliver different bits of functionality. So what typically happens with those, with feature files if your team uses Cucumber or with your test specification classes, if you’re using JUnit or RSpec or whatever.

What typically happens is as you deliver new user stories, you will be expanding your feature, the executable specifications of your feature with every new user story you deliver. Sometimes you’ll be adding additional scenarios. Sometimes you’ll be changing the existing scenarios. Sometimes you’ll be removing scenarios when you see that a certain aspects of a feature don’t gain as much traction as you are hoping it to gain. So again, with every user story your team delivers, you’re basically affecting the current shape of the feature your system has.

Henry Suryawirawan: Wow! I think I get a lot of explanation and clarity on how teams should manage their given-when-then, right. I see some teams actually just write it in the stories and then it gets lost once the story is done, maybe a few sprints before, right. And the teams also couldn’t figure out how to collect back all these given-when-then. So I think all these explanations make sense.

[00:47:09] Screenplay Pattern

Henry Suryawirawan: Both of you have mentioned a few techniques in addition to making BDD work more effectively, right? Things like journey mapping, I think impact mapping, private canvases and all of these are also mentioned in the book. Maybe if you can highlight one or two of your favorites, techniques that we can also explore in order to make our BDD implementation much more effective.

Jan Molak: Yeah, sure thing. So maybe I could start with the screenplay pattern. So the pattern itself caused a bit of a stir when we first talked about it in the test automation community and the BDD community, in general. So what is screenplay pattern? So screenplay pattern is quite an innovative, user centered approach to writing high quality automated acceptance tests. And what it does, it steers you towards an effective use of layers of abstraction and helps your test scenarios capture the business vernacular of your domain. It also encourages good testing and software engineering habits on your team.

Now, how does it do it? You see, typically when we do automated testing, when we write automated tests, we express those tests from the perspective of an integration tool interacting with our system. So in UI testing, we would write a test as a WebDriver. So go to a page, click on a button, enter a value into a field, and so on. When we do API testing, we express it from the point of view, really, of an HTTP client, right? So I send a request, I get the response, I verify a status code.

But what we typically miss in those scenarios is we completely miss the business vocabulary. We miss the business reason behind those scenarios. We completely fail to capture all this information in our test script. So with the screenplay pattern, what we do is we flip this whole thing on its head. So instead of expressing the automated tests from the perspective of a low level tool, we look at them as a way to express all the different tasks that an external user of our system or of our interface would perform in order to accomplish their goal.

So rather than talking about a WebDriver or Playwright or Cypress, clicking on buttons and entering values into fields or talking about REST Assured, we identify actors. So those could be either people or external systems interacting with our system. What we do next is we capture the specific goals and tasks they need to perform in order to accomplish their goals.

So, for example, we might focus our scenario on a person or a persona of a shopper that goes to an e commerce website, tries to find a product, adds it to the basket, makes sure that the tax is calculated correctly and so on. So, what we do with this pattern is we actually capture the business vocabulary, we capture the business intent behind it, given a piece of functionality. And we use high quality code to express those things and capture them in a form of an executable specification.

Now, the interesting thing about the pattern itself is that it’s actually, you know, it might sound like something really complicated, but In fact, it really isn’t. It’s made up of five main building blocks. So the main thing, you know, in an user centered model is a model representing a user or an external system interacting with our system. So we have actors. Actors are the first building block of the screenplay pattern. Now, I already talked about the tasks. So tasks are basically a way to express those different things that an actor would do in order to accomplish their goal. They’re basically aggregates of lower level subtasks or interactions with the system. So we have actors and we have tasks. Those tasks are typically made up of interactions. Now, interactions are the lower level actions that the actor would perform in order to interact with the system, hence the name.

So we have actors who perform tasks, and those tasks are made up of other tasks and interactions. Now, how do those actors perform the tasks? So, the interesting thing about the screenplay pattern is that we try to encapsulate all the lower level API calls to our API clients. So you wouldn’t see any, you know, WebDriver or Playwright or Cypress calls in your tests. All those would be encapsulated in another class. So those classes are called abilities. And abilities are basically thin wrappers around those integration libraries. So we have actors who perform tasks. Tasks are made up of other subtasks and interactions. And what enables those interactions are abilities. And abilities, again, just a thin wrappers around your integration library.

So those are the four elements. Now, the fifth one is questions. Now, similar to the CQRS model for designing system architecture, in the screenplay, we also separate the interactions that the actor would perform from the information they would get out of the system under test. So, an equivalent of a command, you know, from your CQRS model are the interactions in screenplay. Equivalent of a request in the screenplay pattern are questions. So questions are a way to retrieve information from the system. And questions could be things like, well, get me that text from that element on the screen, or get me the last response body that you received from this API request, or get me the position of this button in my mobile app.

So again, you’ve got those five building blocks. So actors, tasks, interactions, abilities, and questions. And out of those five building blocks, you can develop pretty much any type of automated tests you might dream of really. So we’ve used it for UI testing, for mobile testing, for API testing, for performance testing, for visual regression testing.

You can do all sorts of things by expressing your test scenarios using those five building blocks, and also gives you a very nice way to introduce code reusability patterns into your code base as well.

Henry Suryawirawan: Thank you so much for such elaborate explanation about screenplay pattern. I personally, it’s the first time for me to learn about this. I think the abstraction makes sense, the five things that you mentioned.

Maybe if I can ask a little bit further, why was it causing a lot of stir in the testing community? And then secondly, if I’m not mistaken, not just coming up with this pattern, right? You also contribute an open source project called Serenity. Maybe if you can also give a high level about this Serenity library.

John Ferguson Smart: So I can chip in on this side of things, cause I’ve seen a lot of conversations online when we first proposed or came out with it, started talking about screenplay, a lot of people still think in terms of UI as a test automation approach. So a lot of people are very anchored on UI testing being the default way to test an application and sort of goes back to the very first implementations of test automation where it was tools like Mercury and, uh whatever it’s called these days, where it was just record replay tools, so very clunky, basic scripting, literally Visual Basic scripting, uh, style scripting. And it tended to be very ugly and unmaintainable from when you come in from this perspective of a software engineering and you’re trying to make your code maintainable, you’re trying to make it understandable, you’re trying to write code that when you come back to it next week and you have to maintain it or change it, you can understand what it’s doing. I mean, I have to be careful with my own code, because I won’t understand what I did this afternoon if I’m not careful. So looking at code later on, it’s really important to make sure that your code captures the intent. And the traditional way that people tend to do this, even one of the big things that people try to do is use the Page objects approach.

The problem is most people that we see, or I’d say most code that we see implementing the Page object approach becomes very heavyweight and convoluted and complicated and hard to understand and hard to maintain. So you’ll have Page objects that try and represent an entire webpage, which the name might indicate, and that it’ll be a thousand lines long and have a gazillion selectors, and then a whole lot of logic and assertions inside it, and do a whole lot of different stuff, and that means that if a test changes for reason A, and then the test also needs to change for reason B, a different test, they’ll both affect the same Page object, and you get churn and instability. And you might have an implementation for one test that’s done in a particular way, then for another test you need to do something else. And so it can create code that’s historically very hard to maintain.

And so basically, the screenplay approach really is a way of what happens when you apply OO and functional programming principles, just clean coding principles to the problem of test automation. And also, you start to step away from the screen, step away from the UI, and think about, hey, what is the business actually trying to do here? What is the task?

And so I was talking to a team recently, and using an approach that recommended a while back, basically having a very clean separation of the description of your test, the description of what the business is trying to do, what the users are trying to do, and how you implement that. And so they had Cucumber scenarios where it was things like given I want to purchase some books, when I add these books to the shopping cart, then the tax should be calculated correctly. And then, under the hood, they had three different implementations of that that they could hook in based on different tags and glue code or whatnot. One implementation was through the UI, another implementation was through the API, another implementation was just with their pure domain logic, so effectively unit tests. And those three implementations were all driven by the same Cucumber scenarios, in that under the hood it was using actors, but different tasks for those actors. And I think the domain model was just a pure unit test, but the integration, the API calls and the UI calls, the code read very similar. It was an actor attempt to add these items to the shopping cart, except there were two slight variations. One was going through the UI, one was going through the API. And the Cucumber layer never changed. It was the same for both implementations. Under the hood, in the step definitions, the code was very similar. Slightly different because it uses different tasks, but very relatable. You could read it and understand what it was doing. The only real difference was in the implementation of those tasks.

Now that sort of approach would be impossible to do if you start off with the mindset of I’m going to use Page objects. You can only have that sort of level of abstraction, when you think in terms of I’m going to model how my user interacts with the system in business language, then I’m going to decide what the best way to implement that is. What tasks do I need? Does the business, the user need to do, and then how will those tasks actually be performed? And screenplay is not the only way of doing that, but it is a very elegant way of doing that.

[00:58:42] Serenity Project

Henry Suryawirawan: Wow, it sounds really powerful. So I think having a unified Cucumber feature test, and you can have a different implementation detail, so to speak, right? I think that’s really powerful. So yeah, going into the Serenity part, right? So maybe if one of you can also share what kind of cool project that you’re doing with this Serenity library?

Jan Molak: I can take the JavaScript part. So I’m the maintainer of Serenity/JS project. So, in terms of screenplay, just to maybe, you know, build on what John’s already said, what I think what’s really, really important about screenplay is that it helps you focus on the business language and business domain and avoid the implementation details. I mean, if you think about it from the implementation perspective, it’s actually very, very important. Your company, your business, the way your business processes work, those tend to change much less frequently than the software systems that we implement to support them, right?

Now, typically a bank would be doing their business in a very similar way over the last hundred years, right? Or an airline or an automotive company, right? Unless you’re a startup that pivots their product every week or so. Then typically the way the vocabulary you use, the processes you use there, they tend to remain fairly constant. It’s just we constantly rebuild the software systems that support them because new technologies come along. So if you focus on expressing those test scenarios from the perspective of the business, you will see much less change in the overall flow of the scenarios, probably a bit more change on the implementation side or on the integration side.

So now going back to your question, Henry, about Serenity/JS, there’s a lot of interesting things that the Serenity/JS team is doing right now with all the new features. So recently we’ve introduced Serenity/JS 3.0. Now, one interesting feature, I mean, there are many of those, but most interesting feature about Serenity/JS that introduces a universal web facade. What it basically means is that you can create your automated tests using Serenity/JS screenplay pattern building blocks, so interactions and questions, and you can then plug any web integration tool that is currently supported into your tests. So you can run the exact same scenario using WebDriverIO or Playwright or Protractor or Puppeteer, which basically allows you to be completely vendor agnostic. So should there be a better integration tool, you know, in the future, or you need to change from one tool to another, or you want to share your code across different teams that use different tools, you can do it. Again, thanks to the abstraction and thanks for focusing on the business vocabulary.

One other interesting feature that we’ve introduced recently is component testing. So apart from doing end to end web testing that we typically use an acceptance testing framework for, you can also do component testing now.

So you can test individual React or Vue or Svelte or Solid components using Serenity/JS and Prey, right? So you can design and develop your low level interactions and questions, again, screenplay pattern building blocks, you know, in a very confined space of a single component, and then plug them into your end to end tests.

Now, what this allows you to do, it allows you to share test automation code between component teams. and end to end teams or product teams. So, for example, if you have a component team building a design system for a company, you know, building calendar widgets and drop down boxes and so on, they can provide you with test code that interacts with those specific components.

And if you now want to incorporate those components into a larger system and write an end to end test that interacts with those components, you can simply plug in this test automation code that has already been tested at the unit level into your end to end tests. Now, if you think about it, this approach removes completely the problem of trying to figure out, well, what CSS selector should I use to interact with a specific component? Well, you don’t care anymore. You just take a library to interact with a date picker. You don’t care how the date picker is built. This also allows your UI components team to have much greater flexibility and freedom because now they don’t need to worry about all the other teams.

So if you think about it, the great thing about being able to use the same patterns for different kinds of testing, so end to end testing, acceptance testing, performance testing, component testing, is that you actually get to reuse test code across all those different test suites. So you don’t have to do your work, you know, from scratch, you don’t have to start from scratch every single time. Once you have a piece of code that works, you can just plug it into another test suite and run it in a different context. Now, it’s particularly interesting in the context of component testing. So, if you think about it, one, probably a popular pattern that’s very often used across, you know, especially large enterprises is that you would have component teams, UI component teams that develop design systems or UI widgets for other teams to use. So you might have a component team that designs charts or date pickers or drop down boxes or buttons for all the other teams to use. And other teams would assemble those into your booking systems or trading dashboards or other sorts of UI applications. And this way, by reusing the common UI components, they ensure a common look and feel of all those systems, and they also reduce the amount of rework that would be required if they wanted to build all this stuff from scratch themselves.

Now, with the screenplay pattern and Serenity implementation of the screenplay pattern, what you can do is you can share your test code the exact same way you can share your production code. So again, if we have our component team that designs this, you know, date picker, or a drop down box, or a button, and so on, apart from providing other teams with this specific implementation of a component, they can also provide those teams with test code to interact with those components.

So, if you have a date picker, you might have a screenplay task to advance a month, or pick a date or get the current date, enter a value, and so on. Now, another team using this specific widget could now incorporate the widget into the application and incorporate the widget specific test code into their end to end tests.

So again, if you have a trading dashboard, for instance, where you want to set a date of a trade, you don’t have to think about, well, how the date picker is structured. What are the HTML tags I need to use? What are the CSS selectors? No, you don’t have to discover this whole thing from scratch every single time. You don’t have to handcraft it. You can simply reuse the code that the team who developed the widget has already provided you with. If you think about it, this is a complete game changer in test automation because it allows you to avoid, you know, rediscovery of basic things like selectors and structures all over the enterprise and doing it all over again and then seeing all the test suites break if someone changes even a slight thing in a widget.

It allows you to simply reuse test code the same way you would reuse production code, and completely addresses this whole class of problems around how do I find the right selector for this widget? Well, you don’t care. You don’t need to worry about the selectors. You need to care about the business scenario. You need to be able to prove or verify that a trader can book a trade, that a traveler can book a plane ticket, that a shopper can put a thing into their basket. You don’t need to care about the CSS selectors that you need to use to interact with the shopping basket. You simply reuse the test code.

So that’s an absolutely amazing thing and a completely new game changer for many teams that have been using, you know, particularly Serenity/JS and, yeah, I’m actually looking forward to how it gets adopted in the rest of the industry.

Henry Suryawirawan: Thank you so much for, you know, explaining in depth about this Serenity/JS. So I think I will put it in the show notes for people who are interested in this screenplay pattern. That’s the first thing, right? And especially if you have JavaScript project, feel free to try to incorporate this Serenity/JS.

John Ferguson Smart: I’m a big fan of Serenity/JS. I’d use it for any sort of front end project or even front end plus APIs. If you’ve got a team who’s familiar with TypeScript and JavaScript or if the testers are happy to learn that or familiar with that, it really is a no brainer. One of the things I find really cool about that, and we’ll talk about Serenity BDD, but Serenity BDD is basically the Java implementation of the screenplay pattern, and there’s a lot of other stuff that goes with it, but it’s a similar concept in the Java world, so often in teams that I coach, what I’d try to suggest is if they’re doing front end work or UI work, or if they’re comfortable with JavaScript, then they should offer Serenity/JS, because of just the reusable components and the way it fits nicely into the front end build cycles.

And when they’re working on either backend components or sometimes end to end scenarios, if they’re more comfortable with the Java world, then Serenity BDD allows that as well. It integrates with API testing and UI testing with Selenium and Playwright, and does all the nice screenplay stuff so you can create those reusable business components.

But in both cases, what I really find powerful, it’s sort of what Jan was alluding to, is the idea that what screenplay does, it helps you step away from the implementation. And we’ve actually used it. One of the first big screenplay projects we did was with teams where the testers were not particularly experienced with either the domain or with even Java. And so the way we got that team to sort of scale their automation and to produce more automated tests than anyone expected was by getting more experienced developers or testers to write these tasks. So write the task of place a trade or place a booking or create a customer, which actually interact with an API or a UI or a screen or whatever they need to interact with. And then, other testers who are maybe less experienced, they can assemble those tasks, just like Lego blocks, and reuse those tasks without having to worry about what selector to use. And if they need to extend it, or add a new task, they can look at what’s already been done and start to learn how that works. But, they’ve already got this set of building blocks, which is very focused not on how to click on a button, but on how to interact with the business domain.

And that’s something you see a lot of the low code tools these days. None of them go anywhere near that. None of them are able to. The best low code tool in my experience is when you create it yourself and it maps to your domain. All of the low code tools that we’re seeing now, there might be exceptions, but I’m not sure how they could be, are very much focused on just basically scripting out low level interactions. And, if you’ve got a good marketing team, you might be able to sell that well, because it makes it feel easy to click on a button, but that’s not where you get the leverage. Where you get the leverage is where you can reuse business components and model business domains and actually understand that you are testing the important stuff and scaling your tests. So if you have an end to end flow where you onboard a customer of a particular type and may take them through a long KYC process and get them out the other side and make sure you’ve got heaps of systems that interact together correctly, and then you need to do another flow with a different type of customer, you can scale that really well because you have all these reusable components. If you had to, sort of, re-script that, even with low code tools, it’s still quite a headache when you actually start to do things at scale. So I think that reusability, I mean, it does take some planning and some thought and some design and that’s where the BDD aspect helps because it does make you think about modeling the domain concepts, that’s where you get the scalability.

Jan Molak: One more thing to add to that. So people often ask us now, well, so should I choose Serenity/JS or should I choose Serenity BDD? You know, which one’s better? I mean, that’s not really the question that is really important here. So if you think about it now, you can use either. If your project is predominantly using a JVM language, now Serenity BDD is a perfect choice. If it’s a predominantly frontend or scripting languages, Serenity/JS is great. But what’s interesting is that both frameworks use the same reporting engine. So, in fact, you could actually use both Serenity/JS and Serenity BDD on the same project. And many teams do that. So, you know, sometimes you might have a huge monorepo and, you know, some button service uses Serenity BDD, the front end ones using Serenity/JS, but what it can do is you can actually make them produce reports that get combined and result in a unified living documentation of your entire system. So, we’ve come across cases like that, so we’ve made Serenity/JS and Serenity BDD support this case as well.

[01:11:33] Other BDD Techniques

John Ferguson Smart: My other favorite from working with more complex requirements like finance, insurance and whatnot, is feature mapping, which I find a really nice way of mapping out user journeys, mapping out variations of what a user actually does or how data gets transformed. So it might be a user, it might also be a client record. There are lots of different ways of looking at how a process works. So feature mapping is very good at analyzing, walking through, and visualizing how a process actually plays out and what are the goals, what are the outcomes we’re trying to achieve. A little bit more than say story mapping which is much broader, feature mapping is very focused on a particular use case and how we look at the variations around that.

And the close second would be example mapping, which is a sort of go-to tool I find, just for getting clarity on what we’re trying to do. It’s a really very deceptively simple, but very effective technique for mapping out examples, for identifying examples and counter examples and hunting out new ones.

Jan Molak: I think my other personal favorite is impact mapping. And the reason why I like impact mapping is because it helps you to create this very easy to see visual association between the goal you’re trying to accomplish, the actors that can help you accomplish this goal or can, you know, hinder your progress. So it can help you understand the behaviors you want to change and form some hypothesis around how you’re going to change those behaviors. Now, what this allows us to do when using impact mapping tells us to create a very good communication link between typically business sponsors and the delivery team.

Because business sponsors understand the goals very well. They tend to have a very good understanding of what is the audience they’re trying to address with whatever functionality they want to ship. Now, developers have a very good understanding of deliverables, of the features they can build. So what those two parties need to figure out is how actually we can now create those deliverables, how we can create software in order to change the behavior of the groups that we want to affect in order to accomplish our goal. So it’s a really nice way to use when you facilitate workshops now between business and technical people.

Henry Suryawirawan: Thanks for sharing some of these techniques. If people are interested to learn more, I think the book covers some of these techniques in detail, right, including examples and fictitious stories, how this can be applied. And I think I would also suggest people reading a lot more about these techniques, because they can really make the collaboration effort much more productive, I would say, and effective.

[01:14:13] Testing Layers to Apply BDD

Henry Suryawirawan: So, one thing about BDD that I always think people get confused, where we should apply it, right? Because there is a unit test, integration test, API test, UI test, and so many other tests, right? Maybe a bit of wisdom here. Where should BDD be applied? Can it be applied to all of them or maybe, yeah, based on your experience?

John Ferguson Smart: That’s a really good question! And I tend to answer it by saying you want to use what I call the lowest responsible level of automation. And what I mean by that is what is the lowest possible level you can automate and still have confidence that your feature has been implemented. So BDD transcends end-to-end, unit, integration. It can be any of those. Any test can be a BDD test. A BDD test is simply a way of illustrating that a particular business requirement has been implemented. And you could do that with a unit test or you could do that with an end-to-end test. If you take our example of the product catalog, adding a new column, you might have a BDD scenario where you say, well, given Sally is on the shopping search page, when she searches for some dresses, and she should see the following dresses with these color options.

And that, would that be a unit test? Would it be a UI test? Would it be an end-to-end test? An API test? It could be combinations of all of those. Whatever, depending on your architecture will be the most efficient way of testing that. It could be any of those. So it’s not a specific, you don’t implement BDD at a specific level. You implement it at the level which makes the most sense.

Jan Molak: And I think to add to that one. One of the things I really like about this BDD approach, so focusing on the observable behavior of the piece of software you’re actually interacting with, be it the system as a whole, being a component, being a unit or a function and so on, is that you are focusing on the observable outcome. Now, what you can also do with those tests is you can now describe what should a given piece of your software do. So what you’re doing in those tests, you are not simply specifying what the system already does or what the component or a function already does. You’re specifying what it should do.

Now, why is this important? This thing that we all suffer from, that’s called confirmation bias. Which is basically the tendency to seek out and prefer information that, you know, supports our preexisting beliefs. Now, one of the preexisting beliefs that our software developers have is that now our software, the one we just written is perfect, right? So if we write the software and we write the test after that, what we tend to do is we tend to confirm what we’ve already written in the test.

So, for example, I might have a customer class, I create a test that now I can set the address and I can get the street address from this customer. Which is great. So I’ve just now confirmed that my code does what I told it to do. Big achievements, right? This is my gold medal. But what is much more interesting is what are the implications for the system of a customer changing their address, you know, what should happen in the system when the customer changes their address? Should their default settings change? Should they be charged different tax? How should the system behave when we make this change? Now, with BDD, we’re focusing on the outcomes.

So what we can do in our test scenarios? So rather than saying that, you know, my setAddress method works, we’re saying that when a customer changes their address, their default tax rates should change to whatever state they are in right now. Now, this is much more useful from the business perspective as well, because now should this test fail, you know exactly what business functionality is affected. You can now easily judge the impact on your business based on the failure of the test.

If you tell a business person that, well, you know what, I have this unit test failing that was testing this one method in a class that you’ve already forgotten about. I mean, should we now stop the release? The business person will be like, what? What are you talking about? But now if you say, well, actually now with this, those changes we just introduced, we’ve affected the behavior of how we calculate tax for our customers. Well, this will get you a much better answer from a business sponsor around the risk readiness of the system.

Henry Suryawirawan: Thanks for expanding all those, right. So I think I really like the confirmation bias that you mentioned, right? So we, developers, if we have to incorporate new tests in our software, we always try to confirm whatever that we have written, right? So not necessarily the quality will be much better.

[01:18:28] BDD Anti-Patterns

Henry Suryawirawan: So both of you are very experienced in BDD and software testing in general. Even John mentioned in the beginning you have worked with BDD for 20 years. That’s really a long time. Are there anti-patterns, the common anti-patterns that you see, maybe your favorite anti-patterns that you should advise us to avoid? Maybe some of anti-patterns here, if you can mention.

John Ferguson Smart: There are so many. I’d say the biggest one is what we said at the start. Treating BDD as a testing activity. Because if BDD becomes considered as just another way of writing tests, it short circuits the whole BDD conversation process and drains a whole stack of most of its value, in fact, and also makes it actually harder to do what you set out originally to do, which is write the test. If all you need to do is write or is automate tests, you don’t need a BDD tool. You can do that perfectly well with any of the Java or JavaScript or Python testing tools that already exist. You don’t need BDD for that. BDD is all about the collaboration. So the biggest anti-pattern is writing test scripts in Gherkin.

The second big anti-pattern is at the other end of the scale, what we were saying earlier, writing requirements in given-when-then. So using the given-when-then notation as a way to write the requirements before you have the conversation and not as a way to confirm the understanding after the conversation. Because again, that short circuits the conversation that introduces the cognitive biases that Jan was talking about, but from the other direction that you assume that whoever wrote these wonderful given-when-then statements knows what they’re talking about. So you don’t question them. You assume that they’re good. Whereas if you come up with something in a much more tentative and much more lightweight, you can have a much more interesting conversation. So I’d say they’re the two biggest anti-patterns to avoid from my perspective.

Jan Molak: Yeah, absolutely. I think the biggest one, just like John said, is using BDD tools to avoid collaboration. That’s kind of completely missing the point. Now, interestingly enough, this typically happens out of good intentions actually. So you’ll very often see people new to Behavior Driven Development like, you know, product owners or business analysts. They’re trying to ease some of the burden put on the development team, and they just try to write all the Cucumber scenarios before giving them to the developers. Just so that they can avoid all this additional work. So they’re trying to do the good thing, but the outcome is the complete opposite.

Now, you might have testers write their test scenarios in Cucumber so that they’re easier to understand to other people involved in software delivery. But then again, they’ll write them typically from the point of view of a tester. So they produce a test script, you know, full of implementation specific logic. Talking about UI screens and clicking on buttons and so on. And what then typically happens is that, you know, whoever reads this scenario who is not a tester by profession, they’ll just now get bored halfway through if you’re lucky. And they just say, yeah, okay fine. I trust you. Just go with it. Yeah. That’s cool. So again, we are missing the whole collaboration aspect of Behavior Driven Development.

And I think the next one is using UI to test everything. I think that’s one of the more common ones. So very often people think of Behavior Driven Development as using Cucumber to drive, you know, Selenium or Playwright or Cypress or whatever other web testing tool and use that to verify your system.

And again, that’s a misconception. The whole idea of expressing your executable specifications in an implementation agnostic manner is so that you can automate them using the fastest, more sensible interface that makes sense in this particular context. That’s what John was saying earlier about the lowest responsible level. If you can verify your business requirement at the unit level just by interacting with classes and functions, awesome. This way you can avoid incorporating all the infrastructure, you know, databases, Docker containers, all the other stuff just to verify something you can do at a unit level. That’s fine.

BDD is not about using UI automation tools to verify a fully assembled system. It’s about helping you collaborate with the different parties involved in software delivery to understand what’s actually required of your system, why you need to deliver it, and then to help you find the best possible way to automate your requirements. If that’s through the UI, fine, but typically you’ll have other better ways to do it.

Henry Suryawirawan: Wow, thanks for sharing all these anti-patterns. I’m sure people will be able to relate to most of them, if not all, I would say. And I think that’s a really key learnings to end this conversation.

[01:23:01] 4 Tech Lead Wisdom

Henry Suryawirawan: So as we go to the end, I have only one last question for each of you, which is what I call 3 technical leadership wisdom. So you can think of it like advice that you wanna give to us to learn from you. So maybe one of you can share each your wisdom.

John Ferguson Smart: Yep, sure. So, I did come up with three tips that hopefully are useful. The first is that good practices transcend languages and frameworks. So the thing to remember is that whatever technology you choose, it’s less important than how you actually use it and how your teams embrace it. So you don’t get too hung up on a particular technology or tool. Focus on the outcomes you’re trying to achieve.

The second one is think before you code. The time that you invest in understanding a problem and digging into the requirements and challenging your own assumptions pays off multiple times in preventing defects and waste.

And the third gem would be lead by example. I find a lot of people will pontificate, I think is a word. They’ll say what you should be doing and tell people off, cause they’re not doing the best practices. Don’t tell people what to do, show them. And if you show that your own commitment to quality and learning and collaboration, then you’ve got a much better chance of getting your team to follow.

Jan Molak: So maybe I’ll just add one more to this list of three so that we have four. When trying to optimize software delivery processes, I see many organizations, many teams look at factors such as delivery speed. So how quickly can we ship new feature to production? However, what’s much more important is whether those features actually solve the business problems they’ve been intended to solve. It’s much more important to focus on doing the right thing than doing the wrong thing quickly.

Henry Suryawirawan: That’s a perfect wisdom to end our conversation. So thank you so much John and Jan for being here. If people want to connect with you, learn more about all these great practices, right? Is there a place where they can find both of you online?

John Ferguson Smart: LinkedIn is usually the best place to find me. It’s the easiest way to reach out and connect. That’s probably, yeah, that’s probably the easiest way, yeah.

Jan Molak: Absolutely. Same here. So if you’d like to connect with us, if you’d like to learn more about our work, just connect on LinkedIn. I’m Jan Molak. My colleague is John Ferguson Smart. Should be easy to look up. So yeah, hopefully we’ll see you all online.

Henry Suryawirawan: So thank you so much for your time. I really learned a lot about BDD and generally good practice about software testing today.

John Ferguson Smart: Thank you. It’s been a pleasure.

Jan Molak: Thank you so much.

– End –