#58 - Principles for Writing Valuable Unit Tests - Vladimir Khorikov

[00:01:17] Episode Introduction

[00:01:17] Henry Suryawirawan: Hello, everyone. Welcome to another episode of the Tech Lead Journal podcast. Thank you for tuning in and spending your time with me today, listening to this episode. If you haven’t, please follow Tech Lead Journal on your podcast app and social media channels on LinkedIn, Twitter, and Instagram. And if you have been enjoying this podcast, consider supporting the show by subscribing at techleadjournal.dev/patron, and support me to continue producing great content every week.

Unit testing, as a software development best practice, has been around for a long time. Despite the history of it and the major benefits that unit testing brings, from my own experience, I still witness there are still many software engineers who do not understand the concept, and even for some who understand it, they do not practice it in their day-to-day software development work. Another thing that I observe from my own experience is unit testing that is not done optimally, ranging from flaky tests–tests that sometimes pass or sometimes fail randomly that normally just require a rerun, brittle tests–tests that easily break every time we make any change to the code under test, tests without any assertion, or unit test code coverage that has to be mandated from the top of the organization, that could introduce wrong incentives and culture.

In order to spread more knowledge about unit testing and its best practices, for today’s episode, I’m happy to share my conversation with Vladimir Khorikov. Vladimir is the author of “Unit Testing: Principles, Practices, and Patterns” and the founder of Enterprise Craftsmanship blog. In this episode, Vladimir and I discussed in-depth about unit testing. Vladimir broke down the four pillars of unit testing, a very important principles that I find very insightful to adhere for our unit testing practices, and also the anatomy of a good unit tests, as well as mentioned a couple of common unit testing anti-patterns. We also discussed topics such as test driven development, code coverage and other unit testing metrics, test mocking and how to use it properly, and how we can be more pragmatic when writing unit tests.

I really enjoyed my conversation with Vladimir, discussing many things about unit testing, clarifying common misconceptions and anti-patterns, and improving my own understanding of unit testing. And I hope you will find some unit testing insights from this episode as well. Consider helping the show by leaving it a rating and review on your podcast app or comments on the social media channels. Those reviews and comments are one of the best ways to help me get this podcast to reach more listeners. And hopefully they will also benefit from all the contents in this podcast. Let’s go to our episode right after our sponsor message.

[00:04:41] Introduction

[00:04:41] Henry Suryawirawan: Hello, everyone. Welcome back to another new episode of the Tech Lead Journal podcast. Today, I have a guest with me. His name is Vladimir Khorikov. He’s the author of the book “Unit Testing Principles, Practices, and Patterns.” So, today we’ll be talking about unit testing and how to do a unit testing from the best practices, tips and tricks, and what you need to do in order to come up with a good code coverage for your software. Interestingly, Vladimir also is a popular author in Pluralsight. So if you look at his profile in Pluralsight, you’ll see a number of courses that he has done over the number of years, including topics like Domain-Driven Design, CQRS, some C# courses, and things like that. So, really interested to talk a lot about software best practices with you today. In particular about unit testing. Vladimir, welcome to the show.

[00:05:30] Vladimir Khorikov: Thank you. Thank you for having me,

[00:05:32] Career Journey

[00:05:32] Henry Suryawirawan: So Vladimir, maybe in the beginning for people to know more about you, could you introduce yourself? Maybe telling more about your career journey, your highlights and turning points.

[00:05:42] Vladimir Khorikov: Sure. So if we start from the very beginning, I started to program professionally in 2005. About seven years ago, I moved from Russia to the US, and about that same time, I started blogging at EnterpriseCraftsmanship.com. So that was one of the turning points in my career. By that time, I was already pretty established as a professional programmer. But writing on my blog forced me to learn even more than I did before. So, to put my thoughts down, I had to basically learn much more than before. Another turning point for me was when I indeed joined Pluralsight. I did an audition with them. They liked my test course, so to speak, and they accepted me as an author on their platform. Since then, I created several courses, about twelve or something like that. A lot of them were, as you mentioned, about Domain-Driven Design. They also authored a learning path with the same subject. Most of the courses in that learning path are mine, about domain models, CQRS and so on.

A couple of years ago, I also started to think about writing a book. The two topics that were the closest to me were about Domain-Driven Design and also about unit testing. So these are the topics that I enjoyed the most. So I thought that maybe I could write a book on one of these subjects. I thought about DDD for a while, about Domain-Driven Design, but I decided not to write a book on that topic because although I think people like my courses about Domain-Driven Design, I don’t really think that I have a lot of new stuff to say on that topic. There are quite a few of good books already on the market that covered all or most of the stuff that I teach on Pluralsight about Domain-Driven Design, and so I decided not to write about that. But with regards to unit testing, the situation here was different. So there were quite a few of books already, but they all either targeted beginners. They didn’t basically teach you the material that I wanted to teach about unit testing. So basically, there wasn’t a comprehensive material for people who are not beginners, who already mastered the foundational, and who wanted to take the next step towards intermediate or advanced level. And for those people, there weren’t a lot of materials on the web. So I decided to write my own book on that topic.

[00:08:20] Unit Testing

[00:08:20] Henry Suryawirawan: So, it’s been a pleasure to talk about unit testing in particular, especially for you who have been around in the industry for so long and also have practiced a lot of things about these software best practices. So in the beginning, maybe we can recap a little bit. What do you think is the definition of unit test?

[00:08:38] Vladimir Khorikov: There are, I think, two definitions that are applicable here. One is more high level and the other one is more low-level. So the high level definition is that you can say that a unit test is any test that is written by developers, by the same people who write the code that is covered by those unit tests. More low level definition would be something like a unit test is an automated test that covers a single unit of behavior. It does it quite fast, quickly, and also it does it in isolation from other unit tests. So these are the three properties of a unit test. If an automated test doesn’t meet any of these three properties, then it’s not a unit test. It’s an integration test. And then, if we go higher the test pyramid. There’re also end-to-end tests or functional tests, or whatever you may call them. Those are a subset of integration tests and the line between, let’s say, end-to-end tests and integration tests is much more blurry than between a unit test and integration test. But even the line between unit test and integration test is not as clear, and so, sometimes I just put both unit tests and integration tests, and even end-to-end tests into the whole bucket of unit tests. So tests that are written by developers themselves.

[00:10:03] Henry Suryawirawan: So speaking about unit tests versus integration tests, I know there are always this long debate about do you really need unit tests? Can we just do integration tests? What do you think about this argument?

[00:10:15] Vladimir Khorikov: Yeah. This has an understandable point. I would say that I disagree with that point, but it is understandable where it comes from. So I would say that people who argue against unit tests in favor of integration tests only argue from the standpoint of their unit tests being too low value. So they are not valuable enough to keep around in the project. They fail with each refactor, and so they are fragile, they are brittle, and they don’t introduce a lot of protection against potential bugs. And so, from that perspective, I understand why they would say so.

Integration tests, on the other hand, they can be more valuable because they cover larger slice of code. They have a higher probability of catching a regression bug, and they also tend to fail less frequently. They tend to produce less false positives, false alarms. To that point, I would say that it just means that you don’t write unit tests properly. You’re probably writing them in a way that makes them brittle. And in a way that doesn’t provide you a lot of overall value. The better approach here is not to get rid of unit tests altogether, but instead, refactor your unit tests, and make them more valuable.

[00:11:34] The Goal of Unit Testing

[00:11:34] Henry Suryawirawan: So, maybe if you are pro to the argument of unit tests, maybe can you define some of the benefits of goals of actually writing unit tests? Especially for developers who are still not yet into unit test practices?

[00:11:47] Vladimir Khorikov: The main goal of unit testing is enabling sustainable growth of your software project. This is the main goal because as the project grows, the amount of code in that project increases, and with it increases the surface area for potential bugs in that code. So basically, the more code you write, the more probability there is that you introduce a bug in that code. That in turn leads to a situation where you release a lot of bugs with each deployment to production, or you spent so much time manually testing that release that your time to market metric suffers drastically. Unit tests help you to avoid this situation. They act as a safety net that ensures that when you introduce new features in your software, the old features stay operational. So you don’t introduce any regressions in those features. And also, they allow you to refactor your code more freely, because you are not afraid of doing that. So you’re able to maintain the quality of your code, and therefore, you’re able to move faster with more quality code base.

[00:12:55] Test-Driven Development

[00:12:55] Henry Suryawirawan: Speaking about unit tests, a lot of people also associate that with TDD, like tests first, or is it like production code first? Do you have any flavor around which one do you think is best?

[00:13:05] Vladimir Khorikov: Test first versus test last versus code first approaches. I didn’t cover it in the book. But yeah, I do have some opinions on that. So I do believe that TDD, test-driven development or test first approach is valuable, but you need to apply it in the right moment of your project. I would say, the general guideline here would be that when you just start with the project, when you don’t have enough information of what you’re doing, or how domain model would look, or how your project would be structured, this is not a good time to apply test-driven development. Because here, you are in the mode of exploring your problem domain, exploring your possible solutions, and here your tests will only drag you down. At this stage, I would recommend to do some outlining, some prototyping. When you are comfortable enough with your problem domain, and you’re pretty sure that the way you are implementing your domain model is the way you’re going to move with moving forward. When you are sure about that, then you can apply the test-driven development approach. Start with tests, and then basically move forward with this approach.

Test-driven development is especially useful when you already have some code base, and even a test suite, and you need to, let’s say, fix some bug. This is where the test-driven development approach shines, because you are able to first put a test that reproduces the bug itself, and then fix that bug. This way, you are sure that your test is correct because you saw that this test fails for good reason, because there is a bug in your code base. And then fix the bug and then make your test pass. Basically, test-driven development solves two problems. It allows you to check that your test itself is correct. I would say it allows you to do that more easily because with code first approach, you still can do that, but it’s just a little bit harder to do. Because when you write your code first, and then you write your test, in order to test that test, you have to modify your code back into incorrect one to see that your test fails first, and then you modify it back and see that the tests started to pass. So with test-driven development, with the test first approach, you avoid these unnecessary steps because you write your test first, ensure that it fails for good reason, and only then, you fix that test and make it pass. So yeah, that’s the first benefit of TDD is that it helps you to check your tests more easily.

The second benefit of TDD is that it helps you structure your development process. When you have a specific goal in mind, you can introduce a higher level test that checks the specific functionality. And then, you can also introduce lower level test, unit test, when you implement specific pieces of that functionality, and so it provides some structure. So yeah, that’s the second benefit of test-driven development. But even without TDD, you will capture a lot of benefits if you just apply unit testing, even if you write unit tests after you write your code.

[00:16:24] Henry Suryawirawan: Thanks for highlighting all these benefits. I also learned from other people, and also from my experience sometimes, the test first approach could actually lead to better design. Simply because you think from the perspective of the client, or the user of your production code, and then you come with the test first, which then leads you to a better design, maybe more testability in mind and things like that. Do you have any thought around test drives a better design?

[00:16:49] Vladimir Khorikov: Well, I would say, testing the ability to test your code is a necessary, but not sufficient requirement for good code design. Because I would say that it’s a good negative indicator, but it’s a bad, positive one. When your code doesn’t allow you to easily test that code, then it’s a good sign that there was something wrong with that code. It’s structured incorrectly. Maybe you didn’t implement dependency injection properly or something like that. But even if you are able to cover this code with unit tests or with integration tests, even then, it’s not a hundred percent guarantee that your code itself is structured properly. So I would say that it does help with code design, but only partially.

[00:17:35] Code Coverage & Other Successful Metrics

[00:17:35] Henry Suryawirawan: A lot of times, when we write unit tests, we associate it with code coverage. So almost every time when someone is writing unit tests, they will care about code coverage. Maybe you can explain a little bit why the emphasis on code coverage? Or if there are any other successful metrics for people to use when they write unit tests.

[00:17:53] Vladimir Khorikov: Code coverage is a very popular metric because it’s maybe the only metric regarding unit tests that you can apply automatically. So you can use some tooling that will give you some specific number and you can track that number. So that’s basically one of the reasons why people like this metric so much. In terms of its merit, I would say that I would apply the same framework here. The same framework as with the ability to test your code. Code coverage metrics are also (a good) negative indicator, but they are a bad, positive one. That’s because if you have a low coverage number, let’s say, anything below 50 or 40%, that’s a good indication that you don’t test enough. You don’t have enough unit tests. But even if you have a high coverage number, 90% or even 100%, it doesn’t guarantee you that your tests are of good quality. You can still have tests that don’t properly test your code. A contrived example here would be assertion free unit testing. It is when you write tests that execute your production code, but they don’t assert anything. This is a simple and easy way for you to bump up your coverage metrics up to 90 or even 100% without actually trying. Because those tests are, well, basically useless. They will not fail and they don’t check anything.

[00:19:18] Henry Suryawirawan: And you mentioned a little bit about what you should aim for when you start with unit tests, right? So there are three important things that you cover there, which is that the tests should be integrated with your development cycle. And then it should target the most important parts of the code base. And you should get maximum value with minimum maintenance costs. Maybe you want to explain more on these three properties?

[00:19:41] Vladimir Khorikov: Yeah. The only way to benefit from tests is when you use those tests. This is why I put an emphasis on that point, that you should have those unit tests, but integration tests as well, integrated into the development cycle so that you run them as often as possible. Ideally, you should run them after each code change. But of course, this is not always feasible, but it’s the ideal. And then, your unit tests also should target the most important parts of your code base first. Because not all your code base is equally important. There are some parts that are more important for the project, and there are some parts of that are less important. For example, business logic or domain model is the most important part of your project, and this is what you need to cover first when doing unit testing. Some infrastructure code, some maybe utility classes, they might be important, but they are usually not as important as business logic.

The third point here is that you shouldn’t just write tests for the sake of unit testing. You should be pragmatic about it, and you should aim at getting the maximum possible value of those tests with the minimum maintenance costs. That is what I cover in the rest of this book, because this is easier said than done. In the remaining of the book, I cover how exactly you can do that.

[00:21:04] Pragmatic Unit Tests

[00:21:04] Henry Suryawirawan: So when you talk about being pragmatic, right? A lot of people has a certain number of coverage that they would aim for. Some put it pretty high, some put it in the middle. What do you think is a good pragmatic number for code coverage? And then when you say about minimum maintenance costs, right? Can you explain a little bit why some unit tests could actually create a high maintenance cost?

[00:21:26] Vladimir Khorikov: Yeah, that’s a good question. So in terms of the specific coverage number, I wouldn’t say that there is any specific number that is good for any project. I wouldn’t even put a specific coverage number as a target for developers. Because you often can see that there is some coverage number, let’s say 80% or 70% that is put as a target, and your build fails if you don’t achieve that target, or if this number goes below that limit. This is not a very good practice. Because it introduces perverse incentives for your developers. So you may end up in situation where you write a lot of low value tests at that exercise, basically useless code, or code that is not very prone to bugs. Or even as I mentioned earlier, you can just omit some assertions because they are hard to implement, and you just exercise your code for the sake of increasing this coverage number. So, first of all, you shouldn’t fail your build because of some specific coverage number. At most, that should be an alert that alerts your developers, which prompts them to investigate why the coverage number has become so low.

But the rule of thumb here would be, if you can, you should separate your domain model from all other parts of your project. Domain model should have a high coverage number. So this is the only part of your project where you can possibly target some high coverage number. Something like 90%. That would be justified in the vast majority of cases. But you shouldn’t apply these metric blankly in your whole project. That would be counterproductive.

[00:23:07] Henry Suryawirawan: The second question is about maintenance. What do you think like, where are the place where sometimes, in unit tests, you might have a high maintenance cost?

[00:23:15] Vladimir Khorikov: Right. There are several areas, but the most prominent, the most notorious area for tests being low value is when they are brittle. So when people write tests that fail with no good reason after basically each refactoring. And we can talk about this more, because this is one of the pillars of unit testing that I also described in the books. One of the four pillars.

[00:23:40] 4 Pillars of Unit Testing

[00:23:40] Henry Suryawirawan: So maybe let’s move on to that four pillars. Could you describe the different pillars that you have for a good unit test?

[00:23:46] Vladimir Khorikov: So the framework with that, I tried to lay out, and it’s about four properties of a good unit test. Four pillars, as I called it, in the book. They are first, it is protection against bugs. The second one is resilience to refactoring. The third one is fast feedback. And the fourth one is maintainability. So let’s start with the second two, and then we’ll go back to the first two, because the second two are easier to explain.

Fast feedback is pretty self-explanatory. It’s about how fast your tests execute. This metric is important because the faster your tests execute, the quicker you get the feedback if your code stops working properly. This saves you time moving in the wrong direction. You need to just keep in mind that a bug found during development is basically costless to fix. It doesn’t take you anything to fix this bug. But a bug that is found in later stages of development, let’s say QA, or even in production, it requires orders of magnitude more effort to fix the bug found in development. And so, you want to push those bugs as far back to the development stage as possible. Fast unit tests help you do exactly that.

The fourth metric is maintainability, and it’s a function of how easy it is to read your tests and understand them, which itself is a function of basically how large your test is. Because the larger your tests, the harder it is to understand that test. Also, the maintainability metric, the second subcategory of that metric, is how easy it is to maintain dependencies with their test operational. If your test works with out of process dependencies, such as the database or the file system, or any other third-party systems, you will have to maintain those dependencies for that test. You will basically have to make sure that your database resides in the proper state. That your network connectivity is fine and so on and so forth. That also adds up maintainability costs for your tests. So that was the third and the fourth metrics.

The first metric, as I said, is protection against bugs or protection against regressions. It is how likely your test to catch a bug if you introduce that bug in your code base. So that’s basically a function of how much of your code your test executes. The larger that slice of code, the better test executes it, the higher the probability that your test will find a bug in that code base, in that slice of code. Of course, given that your test has all the required assertions. And then, the most controversial, or the least I would say understood property of a good unit test is resilience to refactoring. And that is how likely your test to fail, to turn red, if you refactor the code that this test covers. So another way to describe what this metric is, how likely your tests to raise false alarms or false positives. The false alarm is a situation where your test fails, but the underlying code works fine. This usually happens when you refactor your code because when you refactor, you change some implementation details of your code base. And if your tests are coupled to those implementation details, well, they may basically not like that you change those details, and they will notify you about those details changed. So that’s not how you want your tests to work. You want them to fail only when you change the observable behavior of your code base, not the implementation details of it. And so, this second metric is what is responsible for test brittleness, or test fragility. The better this metric, the less fragile your tests are.

And there are several things that contribute to this metric. In my experience, this metric, resilience to refactoring, the most important metric and it’s the single most important factor that differentiates between good and bad tests. Because nowadays, a lot of developers understand the importance of the first metric, protection against regressions. The third metric of test being fast, and the fourth metric, the tests being highly maintainable. But not a lot of developers understand the importance of the second metric, which is resilience to refactoring. I can only speculate why that is, but my idea is that you don’t need refactoring right away. So if we outline the importance of the first metric, protection against regressions, and the second metric, with time, you will have this situation where protection against bugs is important pretty much from the very beginning of your project. Because you want your tests to catch any bugs from the get go. But with resilience to refactoring, it’s not as important in the beginning because the importance in refactoring is not as immediate either. Because in the beginning of the project, your code doesn’t need a lot of refactorings. It’s only when you introduce more and more features, when your project evolves, you need to do refactoring. You need to start doing constant refactoring of your existing code base to make sure that it reflects your new domain knowledge, your new requirements and so on. And so, because you don’t need refactoring right away, the importance of the second metric, of your tests resilience to that refactoring, is also not immediate. That’s why I think a lot of developers don’t pay too much attention to that metric. But it does become important, as important as the first metric, when the project evolves and when it matures.

Oftentimes, what you can see in a lot of projects is that you write tests, and you are quite happy with those tests, but then when you write quite a bit of code, you have to refactor something. You have to introduce a feature that doesn’t quite fit your existing code base, your existing architecture, and you need to do some modifications to that architecture, to that existing code base. And that oftentimes leads to modifications, not only to your production code base, but to your tests as well. Because they were structured in a way that they are coupled to those implementation details, and when you refactor those implementation details, you basically have to refactor your tests as well. This is an example of a false positive, of a false alarm when your tests raise those alarms, and they are false because it’s not necessarily true that after refactoring, you introduce any bugs in your code base. You might have refactored everything perfectly, and the code is still working, but your tests will still fail even though your code might still work. And so, this is the second metric.

You can also think of the first two metrics, protection against regressions or protection against bugs, and resilience to refactoring, you can think of them as of signal-to-noise ratio. It’s when you have, well, basically a signal and divided by noise. Signal part is what your first metric gives you. It gives you the possibility, the probability of catching any regressions in your code base. The noise part is what resilience to refactoring allows you to minimize. So once again, signal is how good your tests are at catching any bugs, and noise is about how many false alarms do those tests raise while they catch those bugs. Both of these metrics are important. Well, signal is important because it’s obviously important for your tests to be able to catch bugs, because otherwise they’re just useless. But noise is also important because even if your tests are able to catch any bugs in your software, even then, if they raise a lot of noise, a lot of false alarms, then you will not be able to differentiate proper failures from false failures. So you will lose all that signal in the sea of noise that your tests generate. And so, it’s important to maximize both of this metrics, protection against bugs and resilience of tests to refactoring.

[00:31:57] Henry Suryawirawan: Thanks for the in-depth explanation about these four pillars or four metrics of unit tests. I get a sense that some people might still be confused about the second metric, which is resilience to refactoring. I would also assume that when you do refactoring, of course your test will fail, right? Of course, you will need to also change your test code. A few things that I pick up from your explanation just now is that there’s a situation where the test could fail, but the actual production code is actually working fine, which is what you called false positive, and the other one is actually when the test relies too much on implementation details. Could you maybe briefly explain about this for people who might be confused? What do you mean by a test that should not rely too much on implementation detail?

[00:32:38] Vladimir Khorikov: So you want your tests to check the behavior of your code from the end user’s perspective. For example, if you have a class that, well, let’s say it renders some message, and it returns an HTML representation of that message as a result. There are two ways to check that class. First is to check the result in HTML message as a whole. And second, is to check how exactly this class generates that HTML message. This is an example of a good test and a bad test, in the sense of a brittle test. The second version of that test which checks the internal implementation details is brittle because it binds, it couples to those implementation details. It basically insists on a specific implementation of that class. If you change that implementation, that test will fail. There are countless number of equally applicable implementations, usually when you implement classes. That’s why it’s a bad idea to couple to a specific implementation of your class when you unit test that class. Basically, because it will raise too many false alarms and that will inhibit your ability to find bugs in your software because you will not be able to rely on that test when that raises too many false positives. I’m not sure if it answers your question, but maybe we can elaborate on that with further questions.

[00:34:01] Anatomy of a Good Unit Test

[00:34:01] Henry Suryawirawan: Yeah, sure. I get a sense that we’ll cover this more when we talk about mocking later on. But you mentioned a couple of times already that some of the bad practices of doing a unit test, when you have a test with the assertion, and then in terms of lines of code, a long test. Could you probably describe a little bit more about the anatomy of a good unit test?

[00:34:20] Vladimir Khorikov: Yeah. That’s the second chapter where I tried to provide a refresher on how unit test is structured. So the usual structure is Arrange, Act, Assert structure where you have three parts of a unit test. The first one, Arrange is some input we’ll use or test fixtures. The second part invokes the code under test, and the third part asserts that the result of that implication is correct. There are some basic guidelines here of how you should and shouldn’t implement those unit test parts.

The first idea is that you shouldn’t use If statements anywhere in your tests. Because your tests should be as dumb as possible. They basically need to verify only one use case. With those if statements, you diverged from that guideline. You start to check multiple use cases with just one test. This is bad because your unit tests is code as well. If you introduce complexity to that code, it becomes more prone to bugs. That’s something that you want to avoid because you don’t have other set of tests that test these tests. You basically want your tests to be as simple as possible so that they don’t have any bugs in themselves. So that’s the first guideline.

The second guideline. Well, I would say that this is the most important guideline. The second most important guideline is that your tests should be autonomous. They shouldn’t depend on each other. So what you often see is that when you have a class that contains several tests, you may introduce some duplications in those tests. The first inclination of a lot of developers is to extract those duplications into something like class fields or class properties, which you can reuse in those tests. This is understandable, and it is good for production code, but it is not a good practice for your test code. Because another important guideline here, important property of a good test is that they do not depend on each other. This is important because you don’t want your tests to fail when you modify some other tests. You want them to be independent from each other, and you want them to test their own aspects of the production code base independently from each other. So I would say that these two are the most important guidelines when it comes to just some foundational guidelines or basic guidelines.

[00:36:49] Henry Suryawirawan: And I guess the other one is about assertion itself, right? You’ve mentioned like tests without assertion or how about multiple assertions?

[00:36:57] Vladimir Khorikov: Yeah, this is a bit debatable topic. There is an opinion that there was a view of that unit test should contain only one assertion. I disagree with that opinion, and I think that a test can contain multiple assertions. That’s perfectly fine. This view on a test having only one assertion comes from another presupposition, I would say. So, this is the view of that a unit test should only cover a small piece of code. This is also incorrect in my opinion, because it’s not the code that is important for a test coverage. It is a specific unit of behavior that your unit tests should cover. The size of the code that it takes you to implement that behavior is irrelevant here. It may take you one class, one method, or multiple classes. As I said, it shouldn’t matter for your unit tests. Even if your unit of behavior takes multiple classes, you should test that unit of behavior as one unit. So you should create all these multiple classes, then invoke the behavior on those under test, and then you can assert the result of that invocation using those multiple classes. So you can assert part of the result in one class, part of the result in the second class. And so that’s why I believe that multiple assertions per test is perfectly fine.

[00:38:16] Test Mocks

[00:38:16] Henry Suryawirawan: So I think this is a good segue to move to the mocking. Sometimes, you would do these mocking techniques or maybe some other test doubles that people need to do in order to structure the particular class or the particular method in order to have the proper Arrange. The first step of the Arrange, Act and Assert. So could you maybe share us a little bit more about this technique about test doubles, about mocking? And yeah, probably we’ll start from there.

[00:38:41] Vladimir Khorikov: This is another controversial, a pretty controversial topic, of where exactly you should apply mocking. There are two schools of thought here. Two major schools of thought. The first one is the London school. It is also sometimes called mockist school, and it advocates for applying mocks for any dependencies other than immutable dependencies. And the other school is the classical school or classic school and this school is also called Detroit or Chicago school of unit testing. This school is about applying mocking only for out of process dependencies, such as the database, SMTP service, and so on. Mocks is one of the most frequent reasons for your tests being brittle. So they affect the second property of a good unit tests a lot. The second property being resilience to refactoring. That’s because mocks are a good way to cement the way the system under test works with its dependencies. You usually don’t want this communication pattern between the system under test and its dependencies. You don’t want it to be cemented. You don’t want it to be set in stone because that is an implementation detail.

As I said, the London school advocates for using mocks for any mutable dependencies. To provide an example, let’s say that if you test a user class, and this class has another class company as a dependency, and that company class is mutable, then this company class would be a dependency for the system under test, and this dependency would be mutable. And so the London school would advocate for mocking this dependency in tests. So basically for replacing their dependency with the test double. This is a bad approach because, as I said, the particular way of communication between the user and the company has nothing to do with the user of the operation. It has nothing to do with how the end user perceives the result of that operation. So, what matters is the end state of these two classes, of the user class and the company class. How exactly they communicated with each other to achieve that end state is not important at all. The company class, you shouldn’t bind your tests to the specific way these two classes communicate with each other. Otherwise, it would lead to test brittleness. So that’s why I think the London school is incorrect about the use of mocks.

The classical school, as I said, it doesn’t advocate for the use of mocks for all mutable dependencies, but only for those of them that are out of process. For example, the database. The classical school, I would say that it also is incorrect in its treatment of mocks. That’s because just like you shouldn’t mock in process mutable dependencies, you also shouldn’t mock all out of process mutable dependencies. To describe this point, we need to take a step back, and talk about how applications evolve together, and what is the main benefit of mocks? The main benefit of mocks when it comes to unit testing is that, as I said, it allows you to cement the way your system under test communicates with those dependencies. You only want to do that if that communication is part of observable behavior of your software. It is part of observable behavior only when the communication with that dependency is visible to the outside world, for example, to the client who invoked some operation in your system.

So let’s take some example. Now let’s say that you develop an API, and this API works with two out of process dependencies. The first one is the application database, and the second one is a message bus. So the client application invokes your API, and the result of that invocation would be that your application modifies some state in the database, and it also emits a message on the message bus. The question here is which of these two out of process dependencies you should mock, the database and the message bus? You need to look at how your application evolves together with those out of process dependencies.

The main guideline here is that you should enable backwards compatibility when you deploy new versions of your software. And that’s because you cannot deploy your applications simultaneously with other applications that depend on your software. For example, when you develop microservice in your organization, and then there is another microservice that depends on your application, or that reads the messages that you put on the bus. If those microservices are deployed separately, for example, your microservices is developed by your team, and the second microservice is deployed by a separate team in the same organization, you may not be able to deploy them simultaneously. And so when you introduce new versions, you should maintain backward compatibility so that the clients of your software will be able to catch up eventually, and start to use new versions of your interface. But before that, you should still be able to use the old version of that interface. To help you with maintaining that backward compatibility, this is where mocks come helpful. Because if this is their primary goal, they allow you to make sure that the communication pattern between your application and the message bus remains the same. As I said, you cannot change this communication pattern, because if you change the structure of your messages, that your application emits on the bus, then the other applications will not understand those messages. You have to maintain this data contract between those two applications.

This is where mocks will be helpful. They will help you to cement this communication pattern between your application and this message bus. But at the same time, this backward compatibility is not the case for the application database. And that’s because no other application has a direct access to that database. You are able to deploy your code base, your project alongside with database. So you basically deploy the database with your project simultaneously. And so there’s no backward compatibility requirement between your application and the database. That’s why you shouldn’t use mocks when you check your database, because those communications between your application and the database, they are also implementation details. Because if they are not visible to the outside world, this pair of two applications, your application and the database, they essentially act as one system. And they do seem so from the end user perspective. Because the end user doesn’t know that you have a database behind the scenes. The only thing it knows is that you provide some APIs for that end user, and then when it invokes those APIs, the state of some of your data changes, but it doesn’t see the data itself. It can only reach out to the data through your APIs. And so, because in this situation, your system behaves as one with the database, you shouldn’t mock the communications between your application and this database. You only need to check the end result of those communications. So the state of your database after the operation is completed. But again, it’s not the case for the message bus, because communications with the message bus are visible to the outside world. They are visible to the client or other applications who work with your application.

So here, if we go back to the mocking topic from the classical perspective, all out of process dependencies can be subdivided into two categories. The first one is managed dependencies and the second one is unmanaged dependencies. Managed dependencies is basically an application database. It is the communications with which are not visible to the outside world, and you shouldn’t mock those dependencies. You should only mock unmanaged dependencies. This is something like a message bus, a third-party system, something like a payment API, a third-party microservice, or something like an SMTP service. So this is something that you do need to mock, because communications with those dependencies are visible externally. They are observable externally, and so you should maintain backward compatibility with those dependencies.

[00:46:44] Henry Suryawirawan: Thank you for the in-depth explanation, including some examples. You mentioned a lot of things here, especially including like some styles of unit testing, be it like, asserting just the end result or the output. And also like, how do you also check whether the state after a certain operation runs, whether it’s correct? And also communication, right? Like, for example, third party message bus and database.

[00:47:05] Unit Testing Anti-Patterns

[00:47:05] Henry Suryawirawan: So thanks for explaining all this in one go. So, before we move on to the last section of the conversation, which is about three tech lead wisdom, before that maybe if you can give us some anti-patterns or bad practices of unit tests for people to know, be aware of. That’s the first thing, and also for them not to repeat the same mistakes.

[00:47:24] Vladimir Khorikov: Yeah, there are some common widespread anti-patterns when it comes to unit testing. They all flow from these four pillars of a good unit test. They all can be derived from these four properties. But it’s still interesting to reiterate those patterns on their own. Well, just review them because they are quite common, as I said. I would say the most common anti-pattern is unit testing private methods and unit testing private state of your production code base. This is bad because private methods, they are private for a reason. And that reason is usually because your production code base doesn’t need them. So let’s say, if we take a class and it has some public methods and it also has some private methods. Those methods are private because the clients of that class doesn’t need to know about those methods. So those methods are not part of the public API of the observable behavior of that class. By exposing them just for the sake of unit testing, you, by definition, expose implementation details because of those private methods, they are, by definition, implementation details. And so, by binding your tests to those implementation details, you make your tests fragile. You make them more brittle. This is also a consequence of violating the second pillar of a good unit test. So it’s resilience to refactoring. It’s when you bind your tests, you couple them to implementation details instead of observable behavior of the production code.

The second most common anti-pattern is similar. It’s about exposing not methods but state. It is also important to avoid that, basically for the same reasons. Because in your production code, if some class has private state, the state is private also for a reason. It’s because the client of that class don’t need that state to operate. And so, it also becomes implementation detail by definition because there are no clients in the production code that required that state. So when you bind your tests to that private state, you are also violating the second pillar. You are making your tests less resilient to refactoring. Because when you change that implementation detail, let’s say, you modify the way you store that private state, you will have to update your tests as well. And you don’t want to do that. You only want to update your tests when they point out modification in the observability here and not in implementation details. So I would say these are the two most prominent anti-patterns when it comes to unit testing.

[00:49:56] Tech Lead Wisdom

[00:49:56] Henry Suryawirawan: So thank you so much Vladimir for all this knowledge about unit testing, best practices, what to avoid, what to do. So I really enjoyed that, and I learned a lot myself. But before I let you go, normally I have this one question for all my guests, which is for you to share your three technical leadership wisdom so that people can learn from you.

[00:50:13] Vladimir Khorikov: I’m not sure I can come up with three, but I have one. So I would say, put your thoughts in writing. This comes from my personal experience because, as I said, when I started blogging at my blog, I forced myself to learn much more than I learned before. Even though before that, I worked as a programmer for almost 10 years. And still, since I started blogging, I learned much more than 10 years prior to that. This is the main reason why you should put your thoughts in writing because they force you to structure your thoughts, and even if no one reads that stuff. And by the way, I do recommend that you make those thoughts public. Because even if nobody will read your thoughts, it’s still beneficial for yourself because you will be forced to structure your thoughts and to find connections that you didn’t find before. Even if you somehow subconsciously understood some topics, some programming topics, you probably didn’t understand them as deeply as after you start to write about them. So that’s my main advice for everyone.

[00:51:20] Henry Suryawirawan: Just to clarify this, when you say put thoughts in writing, it’s not writing code, right?

[00:51:25] Vladimir Khorikov: Yeah. It’s more about blogging.

[00:51:27] Henry Suryawirawan: So, thanks again, Vladimir. For people who want to connect with you, or follow the discussion about unit testing further, where they can find you?

[00:51:34] Vladimir Khorikov: Yeah, the main touch point is my blog and enterprisecraftsmanship.com. So I recommend my book, “Unit Testing Principles, Practices, and Patterns”. And, by the way, this book gathered the highest rating among all books published by Manning, according to my tech editor. So yeah, definitely recommend you check out the book, and also check out my courses on Pluralsight. They are about domain-driven design, anemic domain model, CQRS, and so on.

[00:52:02] Henry Suryawirawan: So for the Tech Lead Journal podcast listener, I have a special discount for all Manning books. And also for this episode itself, we’ll be giving you one free ebook for giveaway. So make sure to get that book so that you can learn about unit testing, and plus Vladimir said, it’s a highly rated book. So thanks again, Vladimir. I really enjoyed this conversation and see you next time.

[00:52:23] Vladimir Khorikov: Thank you so much. Thanks for having me.

– End –