#184 - Contract Testing Essentials: A Comprehensive Guide - Lewis Prescott

[00:01:23] Introduction

Henry Suryawirawan: Hello, everyone. Welcome back to another new episode of the Tech Lead Journal podcast. Today, I have with me the author of Contract Testing in Action. It’s still in Early Manning release, right? But I think we can talk about the contract testing today. So if you’re into testing and testing techniques, I think today we’ll learn about this technique, which is kind of like getting some tractions because of the microservice trend. And Lewis Prescott is here with me. So thank you so much for your time and looking forward to learn from you about this contract testing.

Lewis Prescott: Thank you so much for having me. It’s a pleasure to be here.

[00:01:56] Career Journey

Henry Suryawirawan: Right. So Lewis, I always love in the beginning to ask my guests to probably share more a little bit about yourself. So if you can mention any highlights or turning points that you think we can learn from you.

Lewis Prescott: Yeah, of course. So, yeah, I started out, I did a degree in psychology. And then I stumbled upon working in software for a small business that worked on like psychometric profiling. So analyzing people’s behaviors, and, obviously, they had the software that went along with that. I worked out that the software side of things was really interesting. And obviously the software testing. And from there I joined a graduate scheme with a testing consultancy who trained me up and gave me the foundations that I still use today. So that was a real big turning point for me, right? Complete career change to what I thought I was going to be doing.

And then that’s really helped me kind of push through my career, worked in lots of managerial roles within testing. Learned to code on the job as part of the consultancy which was completely foreign to me when I started and I, yeah, it took me a long time to get to where I am today. And then, yeah, I’ve worked for some really great businesses, so some startups and also company called ASOS, the online retailer, works in a really great environment there. Kind of cross functional teams, very forward thinking. And then worked in the healthcare space for a long time. And I’ve recently joined IBM. And I now work on the NHS app as a test specialist on there. So yeah, lots of variety in my journey, but yeah, kind of consistently in testing.

Henry Suryawirawan: Thank you for sharing your story. So one thing, if I may follow up, right, from what you shared earlier. So in the beginning, you started studying psychology and you even work on a system that works for psychometric tests. So just wondering how do you actually test this kind of system? Because I mean, like the input is mainly human behaviors and their answers and how do you assess the output?

Lewis Prescott: Yeah, of course. So all of the assessments are based on research. So all of them are validated and verified based on the like personality profiles. I can’t remember the name of them at the moment, but like the, kind of well researched, well backed ones. And then the software side of things with purely people being able to access the form and being able to enter the results. I wasn’t doing any of the like interpretation or any of the research, just purely the software side of things.

Henry Suryawirawan: Right. So I could imagine it might be quite difficult to actually assess whether the accuracy of the calculation or the categorization, right? So I think that must be a challenge altogether.

Lewis Prescott: Leave that to professionals, yeah.

[00:04:57] Problems Contract Testing is Solving

Henry Suryawirawan: Right. So today, we’ll be talking about contract testing. So you’re writing this book at the moment, right? So in the first place, maybe many people might have heard about contract testing or might not have heard about contract testing. I personally have heard about it, but haven’t actually seen it in action in any projects that I have. So in the first place, maybe tell us why contract testing exists, what kind of problems that it tries to solve?

Lewis Prescott: Yeah, absolutely. So my journey was started learning about contract testing, completely foreign concept. Really like found it hard to get my head around it until you realize what the problems it solves. Obviously, most of the people that come to concept testing have familiarity with API testing, integration testing, that kind of background. And so those tests are quite unstable, rely on test environments to be available, integrations to be configured. And all those kind of factors contribute to those tests being flaky or taking a long time to run. So that’s one of the areas where contract testing comes in.

Another kind of area is breaking changes to your API. Knowing who is consuming your API, how they’re consuming it, what changes are going to break that contract. And generally, building software in a way which is backwards compatible. So having that confidence to deliver that software going forwards. So contract testing comes in and it kind of solves that, because it gives you fast feedback. It gives you the versioning of your APIs through the contracts. And yeah, we’ll go into more detail about what contract testing is and stuff. But yeah, they’re the kind of main areas where it really helps you in that software delivery lifecycle.

Henry Suryawirawan: Right. I could imagine last time when I used to work a lot with service oriented architecture or microservices, right? So, I mean, in the beginning, maybe it’s easy when you have one-to-one relationship where you have producer and consumer and any change you can just communicate, right? But I guess since the era of microservices and interconnectedness with, you know, so many SaaS applications or maybe APIs available out there, right? You start to have these permutations of possibilities of API integrations.

[00:07:20] Contract Testing Use Cases

Henry Suryawirawan: And I think one of the challenges that you mentioned, right, is breaking change. Initially I think we all just rely on a contract, maybe, Open API spec that you just distribute around or you just tell somebody, hey, we are going to change this and you need to change that. So I think all these definitely is one of the problems that we foresee when you have a lot of API integrations. So tell us a little bit more how contract testing is being utilized, maybe in this kind of era of microservices.

Lewis Prescott: Yeah. So microservice is definitely like the biggest use case for it. Obviously, the more the complexity goes up and the more integrations you have, then obviously you want to scale those things. And so contract testing comes in and effectively the most similar comparison is when you’re using a mock to mock out your integrations. That’s where contract testing comes in. And it says, okay, you’re interacting with this service in this way. That forms part of the contract and this is what you expect to get back from that service in order to render that on the page or store that information in a database. So the contract is formed from the request and the response. And that forms that contract.

The web application or the mobile application then generates that contract and puts it in a central place. So using the tool Pact, it’s stored in a Pact broker. And then the person who creates the API or is the author or whatever the owner, they then replay that contract on their side. So it’s about forming all of those different contracts with all of the different scenarios that you interact with that integration point. For example, error scenarios, different query params, all those kind of interactions and putting those into contracts and then getting the provider, the person, the owner to replay those contracts. So yeah, it’s passing around contracts which have those expected results in, basically.

Henry Suryawirawan: You said something about mock, right? So I think if I get it correctly, right? So we are actually not testing the actual service that you are calling or maybe you are returning back to, right? So you are actually utilizing mock and something like a replay kind of mechanism where you send a particular request and you expect some kind of a response back.

[00:09:52] Contract Testing Components

Henry Suryawirawan: So, I think one area as well that I learned from what you just said, right, there are many kind of components that exist in a contract testing. You mentioned something about central place or a broker. So maybe tell us a little bit more like what are these components that exist in a contract testing so that we get familiar a little bit more about it?

Lewis Prescott: Yeah, absolutely. So the terminology we use in contract testing is you have a consumer, so that’s your web app, your mobile app, another service, that’s your consumer. The provider is the API or the event mechanism that generates those events. And then, they are the provider. Then you have to store the contracts. So where are you going to store them? That’s called the broker. That’s where they’re all stored. And then there’s another component. Because, as you mentioned, the services aren’t communicating with each other. They’re just communicating via a contract. So then there’s a CLI tool, which allows you to query the broker to say, has this contract been verified? And within the Pact world, that’s called can-I-deploy. So yeah, it’s just a way of querying it and saying, does this contract, has it’s been verified by both parties? Okay, we can release. So they’re the kind of four components.

Henry Suryawirawan: Right. Just to recap a bit, so there’s a consumer, so which is the one who consumes the API, right? The providers, the one who produced the API. There’s a broker, where you store the contracts. Of course, there’s a contract itself, the component, the agreed expectations between both parties. And then the last one you mentioned, there’s a CLI tool that we can utilize to query the broker to check about the contract and the expectations and all that, I guess.

[00:11:42] Shared Understanding of the Expectations

Henry Suryawirawan: So I think in your definition in the book, you also mentioned something that I think is quite interesting to highlight here, right? So contract testing is actually a form of testing where you are verifying two systems have actually the same shared understanding about the expectations. So tell us why this is important to have the shared understanding. I think in the requirements world, it is also mentioned a lot about shared expectations, you know, from the business and also the technology team, right? How does this shared understanding also play a part in the contract testing?

Lewis Prescott: Yeah. So it’s, it’s, it’s super important, right, when you’re dealing with multiple teams, distributed teams, remotely, all that kind of thing that we share the source of truth. And also, while you’re developing software, things can change as you’re developing. So the way that we mentioned about generating contracts is via an Open API specification, which is potentially generated after the code is built. Or you could mock it up before the development starts and share that around. How do you share that around, right? You share it in a wiki page. Or you share it in a central location where in a GitHub repository or something like that. All these things require human intervention, right? It’s, oh, I’ve changed the contract. I need to remember to communicate that. I need to update it in the wiki.

And all these kind of things, like, it’s very easy for humans to make mistakes. And so that’s where contract testing kind of comes in is we’re saying that we know things are going to change. We know that the specifications might change over time. And so we want to capture that and help people debug those issues. Because breaking changes can be difficult to identify. And you may have lots of consumers and they all use different pieces of the API and stuff like that. So that sharing of that contract, it is super important to enable this.

And then, the other scenario, right, is you have teams that are working on two different codebases, frontend and the backend. And they’re writing in two different languages. They’re developing at exactly the same time. So like the rapid changes that are happening, you want that shared understanding to continue over time. And so yeah, creating that central place to store it to enable you to version it and everything like that. That hopefully answer your question.

Henry Suryawirawan: Yeah, certainly. So I think, thanks for explaining that, right? Because it’s very important to realize about this shared understanding, because I think maybe one-to-one, again, it’s quite simple, right? You can just back and forth, converse, oh, this is my expectations, this is the behavior, and this is the response of the output, right? But once you have, when you start to have a permutations, like for example, even multiple versions of the APIs, right? The evolution of the APIs, the backward compatibility. And then once you start having more and more consumers, this is where it gets tricky, right? Because you will not be able to trace all the changes that happens and how it will impact those teams, right? Or those consumers.

[00:15:02] Benefits of Contract Testing

Henry Suryawirawan: So I think contract testing definitely is very interesting. So for people now who understand a little bit about contract testing. So tell us maybe some of the benefits that maybe you have seen in the real world where you actually implemented contract testing. So what are some of the obvious benefits that you can share so that people can understand exactly what kind of value they will get out of this?

Lewis Prescott: Yeah, absolutely. And one of the main ones is that faster feedback. Because you’re not relying on an environment. You can point to the contracts that are potentially not even live yet in an environment and you have control over your own code base, right? The consumer generates the contract from their side, the provider spins up their local environment or however they choose to do it to run the contract tests. So getting that fast feedback early is a huge benefit and independently, right? In that scenario, there’s no dependencies. You control everything. The caveat to that obviously is the provider needs to have their data. They need to set up their data in some way or a mechanism for doing that. And then that may come with the complexities. But yeah, so that fast feedback loop, catching those issues earlier. So not waiting for the changes to go into that integration environment, which can be potentially later on in your cycle.

And one I mentioned earlier, right, about the ability to debug those issues can be difficult when you run your integration tests, because you’re pointing at it from the outside. You get a response back, which is a 500 or a 400. And it just says bad request and you’re like, well, I don’t know what that is. What’s wrong with my request? That kind of thing. So like having that contract in place makes it super easy to replay those on both sides.

So yeah, and another huge benefit of this flipping of integration testing to the consumer is it’s consumer-driven, right? And that means that the users in this scenario, because they’re the users of the API, are the ones creating the tests, right? So in terms of being as lifelike and as close to production as you can get. Then these fit nicely in there, and give that consumer view of how the API is being used. So there are a few benefits that you get out of contract testing.

Henry Suryawirawan: Yeah, so I would also imagine that you can deploy things in a much independent way and confidently as well, right? Because before you actually go to production, you would have verified all the interactions that you have with either your consumer providers, right? You would have verified that all can actually work still, right?

And there’s this tool, can-i-deploy? It’s like a fancy term, right? Can I deploy? Then when it’s green and you can safely assume that everything will just work as it is. Compared to the previous way of working where you have tested everything, you know, like your unit tests, integration tests, E2E tests, all working fine. But then once you deploy to production, right? It’s always the case where something broken, because the integration actually changed the behavior. And then you have to quickly roll back, yeah.

[00:18:29] Contract Testing in Testing Pyramid

Henry Suryawirawan: So I think this is very interesting. So I just mentioned about unit test, integration test, and E2E test. So there’s this testing pyramid concept, right, in the testing world. So what do you think, where does contact testing actually reside in that pyramid, right? Is it unit test? Is it integration test? Or is it E2E test?

Lewis Prescott: Yeah, it’s a great point. And like contract testing kind of sits next to the unit tests. So when you’re generating your contracts on the consumer side, it sits next to the function that makes that request, because it needs to intercept that request. And then on the provider side, it’s a bit more like integration level. Because it pulls down that contract and replays it on their side. So they have a running service. So it’s kind of a mixture of unit and integration. So in the pyramid, it kind of sits in that, just below integration tests and just above unit. So yeah, again, it sits nicely in that way and that you have more control and you get that fast feedback a bit further down the pyramid.

[00:19:37] Contract vs Unit vs Integration Tests

Henry Suryawirawan: So I think it’s in the same spirit of a pyramid, right? So do you think we should have more contract testing versus more integration or E2E test? Or like tell us about this shape or ratio of number of tests.

Lewis Prescott: Yeah, so you should definitely have more contract tests, than integration and end-to-end. But it’s not to say that it replaces those. So you still need integration tests for your configuration, for some of your business logic of this API needs to be available at this performance level and stuff like that. The contract is testing the request and response, and it’s not testing the exact values that come back, right?

So, like there’s still lots of unit tests that you need, and there’s lots of integration. Well, not lots, but some integration that you need. So it’s definitely more, you can fit more scenarios in that contract level, where I talked about the error scenarios, the different query parameters that you’re passing. So there’s lots of different scenarios that can fit into that. And then you still need, some of your integration and end-to-end.

Henry Suryawirawan: Yeah, I just want to highlight one more time what you said, right? So this doesn’t replace integration or E2E test, right? And one thing also important to note, the contract testing doesn’t verify the business logic or the correctness of the output or the value, right? So those things should probably be more in the unit test or maybe integration or the E2E. And also it doesn’t test all the other, maybe, quality attributes like performance, maybe security, whatever that is, right? So please take note about that. And I think I’m sure some people would have tried, you know, like verifying correctness of the behavior in the contract testing as well.

[00:21:28] Contract Testing for Public APIs

Henry Suryawirawan: So one thing about contract testing that I also learned from your book, right, you mentioned that for API providers that have public consumers, you know, like you don’t know actually how many consumers of the API, or maybe third party, external third party that you don’t have control over. Contract testing probably is not suitable for those kind of scenarios. So tell us a little bit more about this. Why that is the case?

Lewis Prescott: Yeah. So as I mentioned, the consumer-driven approach. So what we’ve been discussing so far is the kind of consumer-driven way. And so the person who creates those contracts is the consumer. And so obviously in the public and the kind of third-party integration scenario, you don’t have that input and you can’t drive it that way. Especially with the public way, right? Like you don’t want it driven by the consumer, otherwise you’d have an endless backlog of things, like, it would just get insane. Yeah, so from a consumer-driven perspective, it’s not necessarily the right choice, but there’s still value in contract testing. It just sits in a slightly different flow, which there are solutions for. It’s just not in this consumer-driven methodology.

[00:22:46] Types of Contract Testing

Henry Suryawirawan: Yeah, so you have mentioned a few times now, consumer-driven, consumer-driven, right? So I think many people would probably be puzzled, right? Like what is this consumer-driven? Like maybe now is a good time to actually introduce how many different types of contract testing are actually out there?

Lewis Prescott: Yeah, there’s quite a few different types, and it’s also good to call out that there are other people coining contract testing, but they are doing it at a slightly different level. And it can be confused with like schema based testing, which does that kind of static level stuff. So yeah, it is quite confusing and it can get quite blurry, along with a lot of other terms in the testing space as well. But yeah, there’s two main types in terms of what is offered by Pact and PactFlow.

So consumer-driven, where the consumer creates the contracts to provide and verifies them. And then there’s what we coined in the book, provider-driven contract testing, which is also known as bi-directional, because you can kind of do a combination of both. But yeah, the provider-driven approach is kind of more catered to those public API kind of scenarios where the API can generate contracts, verify their own contracts. So Open API specifications, they verify those with their own testing framework, and then they can publish those results to the broker and then the consumer can then verify those against their own contracts. So the provider doesn’t have to do any additional work and replay anything on their side. They’re just saying, here is our contract. We’ve verified everything in here is true. And you can also verify it via the Pact broker. But that’s only available on PactFlow, which is the software as a service version. And the team there who are many of whom created Pact and are part of that. They also have this PactFlow which stores your contracts and does obviously much more functionality like bi-directional.

Henry Suryawirawan: Right. It seems like in your book, you explain that the provider-driven contract testing, right, is suitable for those who start to learn about contract testing or they come from the legacy of, I don’t know, like things like OpenAPI where you have the spec already and you just publish it to a broker and then consumer can kind of like verify it against the broker whether the behavior is the same. And you mentioned about Pact and PactFlow. Pact itself is like probably the most well known contract testing tool or framework out there, right? The PactFlow is like the SaaS, the product on top of Pact, right? For people who are doing this provider-driven contract test.

[00:25:33] CI/CD Workflow with Contract Testing

Henry Suryawirawan: So maybe after knowing about these two types, I think the first traditional one is actually the consumer-driven one, right? So obviously many people would have a challenge in visualizing how is the workflow like, because you have a consumer, you have a producer, you have a broker in between, right? And both sides also make changes. How is the workflow like? And maybe in the typical CI/CD pipeline manner, maybe if you can just illustrate a little bit so that people also understand like how would they work with this contract testing.

Lewis Prescott: Yeah, absolutely. And the key is what we’re talking about, right? It’s deploying with confidence. So there’s kind of three or four steps to get to that point. So the consumer generates the contracts, publishes those to the broker. The provider then pulls down those contracts, verifies those against the latest version or whatever version is going to be released. And then it goes back to the consumer. So then the consumer will use the can-i-deploy tool to make that check to say, okay, I’m going to release this new version of the contract. The provider has verified it. Can I deploy to an environment to production? And so they’re the core steps that you would have in CI. So they would sit in separate places, right? So the consumer on the consumer side, they would have their contract generation, tag it with this, the version they’re going to release and then publish it.

Then the next step they would have in their pipeline is the can-i-deploy to say, okay, I want to release this now. Can I deploy that? And then they’ve tagged that with the specific environment that they’re going to deploy it to and the specific version that’s going to be deployed. And then on the provider side of things, they would obviously have their tests that run in CI which verify that specific version and publish those results. So they would only be publishing those in CI to make sure that it’s not sitting on a branch or anything like that. So they tag it then with the version of the API, which is they’re pointing to and the environment that that is available in.

And so from the provider side, that’s it, from a CI perspective. Unless they also have consumers, right, which they may have if they’re a service that shares data two ways. So then they would also have a can-i-deploy step.

Yeah, so that’s the kind of cycle of things. So then obviously once the consumer’s ready, they’ll deploy that to production and everything will be tagged and ready.

Henry Suryawirawan: Right. So I can visualize it a bit, right, the workflow. I have some follow up questions because, again, like this is probably new to some people as well. The first is about the can-i-deploy, right, because there’s this gap between the consumer generating the contract and publishing it to the broker. And actually for the provider to actually run their test and verify and update the broker. So I guess in the consumer CI pipeline, right? Does it just wait or continuously poll, you know, like can I deploy? Can I deploy? Or like, what’s the trigger like to actually proceed to the next stage of the pipeline, right? Because you might have, you know, integration tests, E2E tests, and all those things that are also waiting.

Lewis Prescott: Yeah, usually, you would have two separate pipelines, right? One which is building, and then the other one which is deploying. So it would usually be split across those two things. Obviously, if the two are connected together, then you’ll be running that step of the same pipeline just later on, right? And so what is clever about the Pact broker is if the provider hasn’t, if theirs is compatible with previous versions, then that consumer can still deploy, because it’s been verified against the previous version and there’s been no changes which is going to affect that deployment. So it can then say, okay, I’ve got this one, three versions ago, and nothing’s changed since then so you can still deploy this. Or the responses are the same based on different factors. So you can still deploy within that scenario.

But obviously, yeah, if there’s a new version, then the can-i-deploy tool will just return a response to say this has not been verified and then you would need to trigger the other pipeline. But that’s easily done, right? If you’re working in a CI/CD environment, then you can easily trigger another pipeline from yours to get those verifier tests to run or communication, right? Like we talk about this a lot in the book, but the communication is the key to this problem and contract testing facilitates that. If you get the communication right, you don’t need contract testing, because everything works and you don’t have any issues about breaking changes or anything. But it’s that communication facilitator.

So that’s where, yeah, in CI/CD, if it hasn’t been verified yet, or if it fails, then you can jump on a call with that team and start discussing it. So there’s obviously a delay in getting that information. But you should be able to move pretty quickly.

Henry Suryawirawan: So thanks for highlighting about the communication and the agreement, right, between those two teams. So having contract testing doesn’t mean that both parties just don’t talk to each other and just rely on whatever tools that exist, right? So I think communication is still the key. Again, the key point here is about shared understanding, right? And you would probably not get shared understanding if you don’t talk to each other. So I think human aspects of the software development is still key.

So I think that is one question that I have. So the CI, I would imagine that you can proceed with the rest of the stages of the pipeline. But somehow before you deploy to production, you have to join with this can-i-deploy parallel track to actually be confident about your change as well.

[00:31:55] Provider Workflow

Henry Suryawirawan: So the other aspect is at the provider side as well, right? Because now if let’s say you have to support multiple consumers, I would imagine that your pipeline will get triggered by multiple consumers updating their contracts. Is that the case? Or like how does the producer actually work by having all these contracts that they need to serve and verify?

Lewis Prescott: Yeah. Yeah. Potentially, that could be the case. Absolutely. But as we discussed earlier about it all being isolated and working independently, right? There shouldn’t be any dependencies. And obviously, if that pipeline gets queued behind another one, then the verification process will still have occurred, right? So shouldn’t delay anything on the multiple consumer side of things. But the area that I mentioned earlier as well about the data side of things, right? So from a provider’s perspective, the heavy duty lifting is the consumer may be sending you lots of scenarios where the data needs to be in this state and you’ve got a hundred of those scenarios, right? And that’s one of the big hurdles in terms of contract testing, because you have to be able to set up that data as part of your test. And as that’s a limiting factor, really. Obviously, there’s a way around that with sharding your database and things like that. But like, obviously, that requires development time and there’s complexities to it. So yeah, the provider is definitely going to be doing the heavy lifting in that respect and the pipeline will have to accommodate for that.

[00:33:31] Getting the Buy-In

Henry Suryawirawan: Yeah, just by having a better understanding about the workflow now, right, I can actually imagine this can be quite tricky to implement, especially if those two teams are not like close to each other, right? Let’s say they’re different departments, they they they don’t used to work together. So if let’s say only one side wants to implement this, but they cannot convince either the producer or consumer to also do the same, will this still work? Or do you think getting the buy-in is something that is very important before you actually embark on this contract testing?

Lewis Prescott: Yeah, the buy-in is potentially the hardest part of this whole journey. And we talk about that a lot in the first few chapters of the book, is what we’re selling as part of this Contract Testing in Action is not necessarily just the technical side of things, because that is only part of it. And lots of people will say I’ve already got an integration test suite or we already test our schemas using a schema testing tool. So there’s only a narrow window of stuff that we need to cover that’s currently not covered. And that’s absolutely fine view, right? It’s just if you want to seek the benefits that we discussed earlier on, then that’s where you’ll want to implement contract testing. And the problem is that usually teams embark on this journey after they’ve already got these suites of things. So yeah, that upfront selling and getting on board early on definitely facilitates that process. And then if you start to hit these hurdles later on, then you can start implementing bi-directional provider-driven approach which will enable you to convert your integration tests to contract tests.

The selling, again, is part of you need a broker, right? So you need someone to pay for that or someone to host that. Obviously, there’s an open source version which you can host yourselves, but that still has running costs. So you’ve got to be able to sell it, and so that’s why we spend so much time in the first couple of chapters talking about what the benefits are, what problems we’re going to solve for that, and what the solution looks like in terms of that lifecycle of the CI/CD. So you can work out, okay, will this fit into our delivery lifecycle and will this solve the problems that we’re currently having? So yeah, the selling is definitely a big part of that.

[00:36:00] Owning the Broker

Henry Suryawirawan: In your view, who should own the broker, right? Because then you have consumer, you have provider, like who should own it actually?

Lewis Prescott: Yeah, that’s a great question. I mean, it usually falls under DevOps, right? Like that kind of team will usually host it if you’ve got a open source version. Or if you pay for the software as a service version with PactFlow, then it really doesn’t matter who’s owning it. It just comes out of someone’s budget. And obviously, because it’s going to be shared across multiple consumers, it will probably be leadership or someone in a management position will be signing off on it, because it will sit across multiple teams. So yeah.

But what I say to people in terms of that question is, until you start using it and like start seeing the benefits of it, then you’re never going to get started, right? So like, even if you have five brokers to start with because you’ve all set up your own ones, and then you can consolidate them at a later date. Like you’ve got to get started, because otherwise, you’ll just be sat there thinking like, what benefit are you going to get out of this? And also the concept as well. The concept’s hard to understand how it’ll work in practice until you actually get a proof of concept. So yeah, it’s a really good question and one can get asked a lot. But yeah, just get started.

[00:37:19] Pact & Other Toolings

Henry Suryawirawan: So for people who want to get started, I know you have mentioned a lot about Pact. Is Pact the right tool to get started or are there other alternatives out there? If I’m not wrong, I’ve heard something about like Mountebank before. I don’t know whether that’s a contract testing tool, but like tell us maybe what are some of the tools we can explore? Or is Pact actually the de facto standard that we should try to use in the very beginning?

Lewis Prescott: Yes, it’s a really great question. So Pact is like the go-to in terms of it has lots of different language supports and has a core like Ruby base. So they all just build off of that. And so that’s definitely the one that I would start with, if you’re looking for one. But yeah, there’s lots of other places which offer some form of contract testing, like Postman has a version and Mountebank might have a version, I’m not sure. Pact has got an adapter for the mocks, so it can definitely facilitate that. And then you’ve got like Sauce Labs, Testsigma. All these now have contract testing solutions. I read the small print about what it’s actually doing behind the scenes. But, yeah.

And the thing is, you don’t actually need a tool. You could do this with a GitHub folder. Or you could do this locally in a local folder where you verify your request and your expectations against the providers. You just need somewhere to share it. So you don’t actually necessarily need a tool. Obviously, the tool provides lots of shortcuts and it’s been doing it for a long time. So it’s got lots of functionality built in to facilitate that process. But the actual concept of contract testing is just the concept. You could implement it yourselves. You could use any of the tools that are available to you. It’s just time.

Henry Suryawirawan: And communication for sure, right? So don’t forget, so you can use all the tools, but if you don’t communicate, I guess it’s not optimal as well. So I could imagine, yeah, I mean, Pact probably will help in terms of workflow and automation, right, for some of those steps. But I would imagine you can also do it manually, although yes, it’s a bit tricky, you need to coordinate and you have to put the things into the right place and triggers your pipeline and things like that.

[00:39:34] Versioning Contracts

Henry Suryawirawan: So I think definitely for people, thanks for the tips, right? Maybe Pact is something that you should explore. And I think Pact also support some of the advanced use cases, right? So you mentioned in your book, something like can-i-deploy, maybe webhook, right? And also about this versioning. So tell us a little bit more, what kind of versioning support that maybe Pact supports and why is it important?

Lewis Prescott: Yeah, so we discussed this a bit earlier when we were talking about that lifecycle view of when you’re deploying it, what environment you’re deploying it to, and all that kind of thing. So obviously, whenever you generate a new contract, that generates a new version. And whenever the provider verifies it, that verifies against that version. And if provider makes changes, then they will update their version as well. So all the versions are stored within the contract. So you wouldn’t necessarily be changing your version of your API, if you don’t have versioning built in.

And on the consumer side of things, you don’t generally have a version for I’m changing my interactions, you just have I’m releasing a new version of the web app. You don’t have that physical versioning. So that’s built in. And so the versions are then supported by tags. So if you’re developing something and it’s just part of a PR or it’s on a branch, then you would tag it with that. And so then you can test new features against what’s in production, because the provider’s tagged a specific version in the broker with production. So then you can point your feature branch to the production version of the API, and then the API might move forwards. But they might not release that to production until later on.

So now you’ve got that matrix and the Pact broker provides a matrix. So you can see, okay, which versions of the consumer are verified with which versions of the provider. And it gives you that nice visual. And so you can develop in isolation without having to rely on that communication of, okay, I’ve deployed this version. I’m going to deploy it there for an hour. So you can test it and then we’ll switch back to the production version, so that we don’t break any of the other consumers. So yeah, it’s that versioning which becomes very complicated, very quickly in a microservices environment.

Henry Suryawirawan: Right. So yes, this is probably something that Pact also helps you out of the box, right? So if you have to support multiple versions, you don’t have to maintain your own matrix, right? All these permutations of different versions.

[00:42:15] Consumer-Driven Contract Testing

Henry Suryawirawan: So I’m actually still intrigued about this consumer-driven, right? Traditionally, in the past, the provider is the king, right? They will just tell everyone, okay, this is my API, you use it or whatever, right? So obviously, normally there’s no room for kind of like the consumer to drive the API. So how come this time is kind of like flipped, right? I mean in a consumer-driven contract testing, the consumer is the king, right? They can actually shape the contract and the provider has to follow them. So tell us why this interesting, I don’t know, evolution in terms of, you know, the power between the API provider and the consumer. So why this makes more sense instead of the traditional one?

Lewis Prescott: Yes. A really great point. So the, um, it kind of comes from Postel’s law. Postel, however you pronounce it. So it’s “Be conservative in what you do, be liberal in what you accept from others.” So it’s kind of come from that concept. And the idea is putting those users first. So when we’re developing our APIs, there’s often in requirements, it’s like, okay, this consumer needs this information in this format. And so it’s being driven from there. It’s like, okay, the consumer needs this information. So how do we put that into our API? And then it’s like, okay, this consumer needs it in this format. Okay, well maybe that doesn’t sit inside this API, because this is for this purpose. And so it’s trying to narrow down those requirements and trying to make it purely based on what’s going to be consumed.

Another example of that, right, is how often do you have information within the API which is never used? Obviously, you can see within, you can ask the other teams, like, are you using these data points? Or in the logs, you could potentially see, okay, we filter the data by this, but no one ever filters the data by that. And so it’s like also enabling you through the data to say, okay, these are the only areas which are being used. So maybe we can reduce down our API, or maybe we need to split it in two, because we’re getting lots of traffic and they’re requesting this filter, which is really expensive. So let’s just put that into another API. So it’s trying to have a data driven approach to breaking down those requirements to what do we actually need to provide. So it makes, yeah, it makes a lot of sense.

And then as you said about like the power of the provider, right, is usually, the provider will create their tests, which will say the API returns this. And then when they make a change, they will update their own tests. How often do we say like don’t test your own code? And it’s like, well, why don’t we do the same with APIs? Because the consumer should be the one testing it, because they’re the ones that rely on it, right? So yeah, there’s lots of examples that I’ve had in my career, right, where you just change your own tests because we did make a change so we know it’s broken. So we change it and then we’ve broken the integration. So yeah, that’s where kind of the consumer-driven approach comes from.

Henry Suryawirawan: Right. I would also imagine maybe sometimes, I don’t know whether this is true, right? You can correct me if I’m wrong, right? So even though the consumer generates the contract, the provider can still negotiate, right? So it’s not like every different version of contract generated by consumers, they will just satisfy, right? So again, communication is the key. Is that correct, by the way?

Lewis Prescott: Yeah, that’s a really really good point. So obviously, if there’s a failure and the provider fails, because the consumer is asking something ridiculous or even asking something small, right? Like, date format, like, we return it in this and you’re expecting it in this. We can’t return it in multiple different formats, that’s impossible. Or yeah, if there’s a specific null value or like there’s a empty or whatever, it has to be a consistent way to do that. So the provider has to say at certain points, we cannot do that. And so the provider still has lots of power. But it’s just driven by that communication of, okay, we would like it like this. How can we come to a solution?

Henry Suryawirawan: Right. Thanks for clarifying that. I think it’s a very interesting take for those people who want to try this contract testing, right?

[00:46:53] Contract Testing for Event-Driven Architecture

Henry Suryawirawan: I think in this era, not just REST APIs or HTTP APIs that people normally use for integrations. So there’s this asynchronous communication as well or message-driven or event-driven architecture, right? Does contract testing help for that kind of scenario? And if so, how does this contract testing now play a part in that kind of architecture?

Lewis Prescott: Yeah, absolutely. So event-driven is another one of the ones that I recommend for people who are getting started with contract testing, because an event is like a smallest section of data and it’s like a small piece of communication. And so it’s a really nice way to get started, because the parameters are a bit less and often you’re just passing around, right? Like sometimes the service isn’t even storing anything or passing it into another system. So event-driven is definitely supported within contract testing. The message part is all we care about, right? So it’s not tied to any tool or any event -driven architecture kind of facilitator. So like Kafka or AWS SNS, anything like that. It’s not tied to anything, it’s just the message, right? So if your events are heavily tied to those and you can’t separate the contract message from that, then contract testing doesn’t work. But if you’ve got a good way of constructing your event message, then you can definitely verify that that contract is supported.

Henry Suryawirawan: So I think in EDA, you will also have this concept of producer and consumer, right? So is it still the case that the consumer is the one driving the event? Because I would imagine it’s the producer that kind of like push the message, right? So is this still the case for the event-driven?

Lewis Prescott: Yeah, I always get this so confused, it’s so confusing. But yeah, it’s flipped on the event driven, right? So the producer is the consumer, and the consumer is the provider. But check the book because it’s definitely right. um, But no, yeah, it’s flipped, obviously. The terminology is very confusing. But the producer generates the contract and then the consumer is verifying it.

Henry Suryawirawan: Interesting. So okay. So maybe we’ll use the different term publisher and subscriber so that you don’t get confused. Right. So thank you so much for this overview about contract testing. I think for people who deal with a lot of integrations, this definitely is one of the tools that you have to utilize in your, I don’t know, CI/CD pipeline or development process so that you can get the confidence to deploy your integration changes, right?

[00:49:40] 3 Tech Lead Wisdom

Henry Suryawirawan: As we reach the end of our conversation, I have one last question for you, Lewis which I call the three technical leadership wisdom. So if you can think of it just like an advice that you want to give to us the listeners here, maybe if you can share what is the version of your three technical leadership wisdom?

Lewis Prescott: Yeah. Thank you. Yeah. Hopefully, what I’ve spoken about makes sense. But yeah. The theme throughout the conversation has been communication. So that’s definitely my number one is be very open with that communication and keep those avenues of communication open. And I do think that the large majority of software delivery is about communication. So that would definitely be my number one.

My number two is breaking down those barriers of quality. So dev and test is one thing, right? Like at the end of the day, we’re trying to deliver a quality product at the end. So breaking down those barriers and making sure that everyone has that quality mindset built in from the start. So that would be my number two.

And then, yeah, my third one would just be to keep a kind of general holistic view of software development. So I’ve always been a generalist. I have my fingers in a lot of pies. But I think that’s really helped me, especially in leadership, is having that holistic view and being able to contribute to lots of different conversations. So yeah, that would be my three.

Henry Suryawirawan: Thanks for sharing that. I like the second one, right? So remove, break down all the barriers for quality result or quality mindset, right? So I think this is key still in any software development team, right? You want to produce product of high quality, right? And contract testing definitely is one tool that you can explore for that.

So for people who would love to learn more about contract testing or maybe they are waiting for your book or maybe a place where they can find you online, is there a place for them to find you?

Lewis Prescott: Yeah, absolutely. Obviously, you can go to Manning and search for Contract Testing in Action. There’s live discussion on there. And then you can find me on LinkedIn, Lewis Prescott. And also on my website pactman.co.uk. And we should also mention that the co-author of my book, Marie Cruz, is having a baby at the moment, so she couldn’t be here today. Unfortunately. But, obviously, yeah, you can connect with her as well on LinkedIn. And we’re more than happy to answer any questions that you have on contract testing.

Henry Suryawirawan: Right. So let me put that all in the show notes. So thank you again for your time today, Lewis. So I hope people learn about contract testing and they get excited to actually give it a try. So thanks again for your time.

Lewis Prescott: Thank you so much. Thank you.

– End –