#71 - Strategic Monoliths and Microservices - Vaughn Vernon

[00:01:18] Episode Introduction

[00:01:18] Henry Suryawirawan: Hello to all of you, my friends and listeners. Welcome to the first new episode of the Tech Lead Journal podcast in 2022! I hope you had a good end of your break, got recharged, and now back with a good amount of energy and clarity to start the new year strong. Tech Lead Journal would like to wish you Happy New Year and may all your aspirations come true this year.

By the way, if you’re new to Tech Lead Journal, I invite you to subscribe and follow the show on your podcast app and our growing social media communities on LinkedIn, Twitter, and Instagram. And if you have been regularly listening and enjoying this podcast and its contents, will you support the show by subscribing as a patron at techleadjournal.dev /patron, and support my journey to continue producing great episodes every week.

My guest for today’s episode is none other than Vaughn Vernon. He’s the leading expert in Domain-Driven Design and reactive software development. Vaughn recently co-authored and published his new book, “Strategic Monoliths and Microservices”, and it is another great addition to his own Vaughn Vernon Signature Series published by Addison Wesley.

This is Vaughn’s second appearance in Tech Lead Journal podcast, and a little bit of fun fact, he also kickstarted the podcast in 2021 with episode 21, in which we discussed Domain-Driven Design and event-driven architecture, literally just one year ago, 50 episodes apart.

In this episode, Vaughn shared his story and rationale for writing the new book and why he thinks it is also important to include the executives as the readers of the book. In his book, Vaughn highly emphasized the importance of focusing on strategic innovative aspects of software development. And for driving those innovations using purposeful architectures. Vaughn then shared his insightful perspective on Conway’s Law, why it is such prevalent these days and very important to understand in software development, and why he compares it with the law of gravity.

Our conversation then continued to discuss two important architectural aspects covered in the book, which are events first architecture and embracing latency, and why they are actually natural to how people communicate and get things done in real life, but surprisingly not so well understood by many software developers out there. Towards the end, Vaughn summed up his book and left an important piece of advice that he wanted to convey regarding monoliths versus microservices, and why in his view, software should require more balance and demand a better strategy.

I always enjoyed my conversation with Vaughn. There are always great new insights that I got from our conversation. This time, around strategic innovations and purposeful architectures, and our interesting conversation about Conway’s Law. If you also enjoy and find this episode useful, I encourage you to share it to someone you know who would also benefit from it. You can also leave a rating and review on your podcast app or share some comments on the social media platform on what you enjoy from this episode. So let’s get our episode started right after our sponsor message.

[00:05:26] Introduction

[00:05:26] Henry Suryawirawan: Hello everyone. Welcome back to another new episode of the Tech Lead Journal podcast. Today, we have a guest who had appeared before in Tech Lead Journal podcast. His previous episode is still one of the top most listened episode in Tech Lead Journal podcast. His name is none other than Vaughn Vernon. So Vaughn just recently published a new book called “Strategic Monoliths and Microservices”. And today we’ll be covering a lot about that, and why you should choose probably either monolith versus microservices? When should you migrate from monolith to microservices? And is microservices always the better option out there? So we’ll be covering all about that.

Vaughn also previously wrote the famous DDD books. Also known as the red book and the green book, “Domain-Driven Design Distilled” and “Implementing Domain Driven Design”. So definitely Vaughn is one of the thought leader in domain-driven design and event-driven architecture and things like that. So welcome back, Vaughn. Really pleased to have you here, and I hope we have a great conversation today.

[00:06:19] Vaughn Vernon: Thank you, Henry. I really enjoyed last year’s conversation. I think we are very near, almost like a year from the previous recording. It’s so nice to be back with you. You’re a very pleasant person to speak with. So it’s my honor to be here.

[00:06:32] Strategic Monoliths and Microservices Book

[00:06:32] Henry Suryawirawan: Thanks for that. So, Vaughn, last time I began the interview by asking you about career journey. I guess we won’t do that again this time. But more about the book itself. What makes you wrote the book, and co-author it with another author? Is there any particular reason, problem, that you would want to convey the message for people by reading the book?

[00:06:51] Vaughn Vernon: Well, I’ll say this. When I wrote “Implementing Domain-Driven Design,” from my viewpoint, it seemed best to demonstrate the bounded contexts in terms of what we would now call microservices. Frankly, I don’t recall hearing about microservices until after the book was published. So maybe I would have thought of it more as service-oriented architecture. I definitely didn’t have that motivation, but that was sort of the flavor of the day. So if we were to talk about a service as architecture, that’s probably how I would have fit it in. But to me, it seemed like a distributed architecture kind of situation, event-driven. And so, I demonstrated that concept in the book. And then, microservices shortly after that became more prevalent, heard a lot about it, and I started to kind of get confused between the various definitions of microservices. Anywhere from a good microservice is about a hundred lines of code, up to a good microservice is about a thousand lines of code, and several stops in between those two. It seemed odd to me that those kinds of sort of boundaries were being drawn because it just seems so arbitrary to me.

Anyway, so by the time I wrote Distilled, microservices was, I would say, well-founded. But in the chapter one, when I just mentioned a little bit about architecture and the Hexagonal or Ports and Adapters architecture, I said that you could view a bounded context as a microservice. It seemed like a good boundary for that. And the definition sort of the size of the microservice would be the size of the ubiquitous language. So that made sense to me. I also knew around then, or maybe earlier, that Sam Newman viewed microservice as a bounded context in his books. So that seemed in harmony, and then also in Distilled, I did say that if there were specific reasons why you needed to divide a bounded context into smaller deployment units, it still is a microservices architecture, but you have different influences or reasons to break up an already rather small concept of a bounded context.

So, as time went along, I noticed that so many companies are just jumping on microservices. To me, it didn’t really seem to make much sense for some of them, because I didn’t think that they really needed microservices. They were small companies, a lot of times startups, and they thought somehow that creating microservices would kind of force them or discipline them to keep things separated. But in reality, a lot of them simply made REST calls, if you will, or REST requests across bounded contexts, and some of the clients that I worked directly with had a lot of problems with that because they would have committed transaction in one microservice, if you will. And then that microservice would call out using a REST request to another microservice to then try to harmonize those two, make the two or more transactions consistent and eventually consistent. But some of those would fail. They literally had teams of people who every day would spend all day long patching the database and trying to figure out like, what worked? What didn’t work? Why did this fail? Oh, our data is inconsistent here. And so, they were constantly doing this, and I just thought, wow, this is such a backward way of doing system architecture altogether. Don’t they understand that if you took this event-driven approach, that it would be a much more reliable, fault-tolerant kind of a system?

So all of those thoughts together, and kind of seeing companies get hurt, I guess, by microservices. I thought, we really need more balance in this. And then I heard topics come up like, oh, macroservices, or right-sized services, or whatever it was. And that just basically meant, well, whatever seems right, right now, that’s going to be our macroservice. And it just, all these thoughts, well, look, it’s a microservice or it’s a monolith. It might be a small monolith and it might be a slightly larger microservice than whatever these other definitions of those things are. So I just wanted to kind of express the idea that don’t be ashamed of a monolith, right? I mean, monolith became such a bad word. I don’t think monolith has ever really been a bad concept. It’s the big ball of mud monolith that is the problem. So yeah, a lot of these thoughts bounced around in my mind. Of course, I’ve been working with Tomasz Jaskula, my co-author for about three years now. I think we just had kind of an anniversary of working together, which took a bit of time to kind of get us both working together finally, and so I asked him if he would like to write this book with me and he was into it. So we did it. It just seemed to me like the industry needed some mature voice to say, it’s okay if you write, if you create a monolith, it’s not uncool. In fact, it may be the coolest thing you could possibly do for your situation.

[00:11:59] Henry Suryawirawan: Thanks for sharing the thought process and the story. So, we have seen it in the industry itself, right? So it went from traditionally, it starts with monolith, and people went into service-oriented architecture, and suddenly, like you said, we went into this microservices craze. Lately, in the last few years, we can see that a lot of people, maybe they had bad experience with microservices. Pulling back themselves from those architecture and into more monolith or maybe a bounded context based microservices. It’s not an easy choice for many people, but there’s a thought process behind it, which is what you want to say in this book.

[00:12:31] Who Should Read the Book

[00:12:31] Henry Suryawirawan: But first of all, this book, when I read it, you mentioned this is not for the techy techies, right? It’s not like for the implementation of whether it’s monolith or microservice. But it’s actually written for the executives or maybe the CTOs, the leaders out there. Why do you think it’s better to write this book from that angle rather than from the technical implementation?

[00:12:51] Vaughn Vernon: Well, first of all, let me say that I think that the book is actually written for a range. And I think that it is very applicable for even young software developers. So there’s nothing in the book that would be a turnoff to very technically thinking people. But I wanted to include the executives, because they’re the ones who are making decisions, and they’re the ones being told by developers. This is what you should do and what you shouldn’t do. And I actually just want them to be informed and to be able to engage logically and methodically with a team or a large organization, who’s kind of saying we should go in this direction. And I just want to say, look, architecture has to have a purpose. If you are just using architecture as a way to be cool or a way to work with techniques and patterns that you’ve never used before, because you would simply like to experiment, that’s unfair for the organization that’s paying for it. Because they’re really paying for your personal satisfaction instead of paying you to create something that is going to be long-lasting and malleable, able to change and be resilient over time. That’s really where our focus should be.

And so, if I help even technical people say, to think this is not an uncool thing to use a monolith. In fact, it’s a very responsible thing to do. Also, focus your thoughts, your curiosities on going beyond the obvious business solutions, and finding really innovative ways of developing software that will add more value to the business. These are the things that will satisfy your need for complexity instead of just tinkering with architectures and new tools. So to me, it’s like kind of level setting for executives. Whether it’s C-suite or senior vice presidents, vice-presidents, directors, managers of software development teams. And then, the chief architect, the architect, the senior architect, the enterprise architect. Even to, I would say, even a brand new software developer should read this and take it in, and don’t let anybody tell them that, “Oh, you should really try this, or you should be using AWS Lambda.” Okay. If you should be using AWS Lambda, then use AWS Lambda. But if someone’s just telling you that, because, “Hey, you got to find a way to do this because it’s so cool”. That’s not actually cool. So I think it’s for everyone.

Executives, they’re really different people these days. You know, when I started working in the industry in 1983, executives and software were wearing three-piece suits, pinstripes and the whole IBM or EDS look and so forth, that was the environment I was in. It was an insurance environment. Not an insurance company, but an insurance services organization, providing software as a service really is what it amounted to, for like claims clearinghouse. That was sort of this environment. Sometimes I think, “Wow, you know, these CTOs, maybe they didn’t grow up with computers”. No, that’s different now. I mean, there are some still in that kind of situation. But really, the executives are pretty savvy technical people these days. And I really just wanted to give them a leg up where they’re often thinking about, I guess, the product or funding or whatever it is, making sure the company stays in business. That maybe they’re distracted by those things, to the extent that they really are not well-informed about software architecture and the innovation process. So I just wanted to give them kind of this brief message that they could read even half of each chapter and get a great idea of what their development teams should be thinking about.

[00:16:48] Strategic Learning and Experimentation

[00:16:48] Henry Suryawirawan: Thanks for clarifying that. Actually, I read the book as well in preparation for this interview, and it’s really relevant for all different types of people. The way I see it, it’s like the green book last time. So when you want to learn about domain-driven design, that’s the " Domain-Driven Design Distilled" book where you can see it from the beginning, why you should choose domain-driven design, and how to implement it. The same way here, like you have the monolith and microservices, but you actually started not from a technical related discussion. You actually started from the business perspective. So part one is about strategic learning and experimentation. I saw this one particular quote, which I think is very interesting for us to talk about. So you said in the book that the ultimate goal is to develop strategically significant software that produces breakthrough innovations rather than what you have touched on, a steady stream of cool technologies or whatever that is. So tell us more about why you started the book in this way?

[00:17:39] Vaughn Vernon: I do want to say Henry, that you’re correct in that the book is very unique in kind of addressing C-level executives and other executives head on, where we’re saying, okay, this book is for you. It is a technology book, but it is for you. And that is different from most technology books. So yes, you are absolutely correct. And then, from the standpoint of why did we take this strategic approach is, well, strategy is what earns. Strategy is what has set companies, organizations apart from others for as long as we have history about that. Inventors have done magnificent things and even inventors who couldn’t realize the concepts of their inventions, like Leonardo da Vinci. I mean, he had just some amazing ideas back in his day, but he couldn’t realize those because the technologies at the time didn’t support his hundreds of years forward-thinking.

But still, even if you take Thomas Edison, Alexander Graham Bell, and not to show any disrespect to any others around the world who had these sorts of groundbreaking inventions, it’s simply the ones that I’m more aware of because of my background in the United States. But these people really did some remarkable things. And Thomas Edison was a great salesman too, and even Alexander Graham Bell, because if you listen to them, they would tell you that you would be convinced that Thomas Edison invented the light bulb, and that’s generally what he’s known for. But he didn’t invent the light bulb. He invented a filament. It had a much longer lifetime than other previous light bulbs. And so, from that standpoint, he was quite innovative and an inventor, but he didn’t invent the light bulb. He improved on the light bulb. There is a story involved, but the thrust of the book is to say, look, if you think that you’re going to really invent something entirely new, that’s going to be really hard. It’s not that you won’t. It’s just that everything it seems has been done before. Of course, there are some things that haven’t. But the point is we can often make groundbreaking improvements on technologies that already exist. To change the way people use certain things to make it better, a better user experience to make their outcomes better, to make them more productive.

This is where I think the energy should be instead of just a developer jumping from one company to the next sort of reimplementing the same architectures and so forth, and then just settling for the ordinary use cases. Isn’t that amazing? Like, you’ve created one of these event-driven architectures or something before, now your experience should be telling you that’s actually sort of the easy part. I already know how to do that. The really hard thing is to make software that solves a different set of problems than previous software. You know what I’m saying? And I have to just say one more thing. It’s funny to me that when you look at want ads for programmers, right? Or software developers, architects, whatever. And it says five plus years previous experience in banking or in stock trades, in insurance, in medical or healthcare. Okay. Why? Seriously, why? Do they want me or you to be the domain expert? Are they actually saying, just bring the knowledge that you had from the other company and come over here and give us the same thing. Well, that’s not innovation at all. That’s copycat. If we keep that copycat or cookie cutter thing going around, nobody has really benefiting more from software than they did those five prior years of that experience. And now we’re just carrying that forward and forward. I think that each of us should really be much more focused on the innovation, the innovative aspects of software development.

[00:21:35] Henry Suryawirawan: Yeah. I tend to agree that these days people jump into technologies. What are the new thing, recent things, new inventions, new frameworks, new platforms, whatever that is. But at the essence of any companies or any kind of product oriented teams is actually to build breakthrough innovations or doing more in experimentation in order to find the real values for the users that they are building the product for. So be it like helping their productivity, cost efficiency, whatever that is, rather than the angle of the next best architecture or the next best technology.

[00:22:04] Purposeful Architecture

[00:22:04] Henry Suryawirawan: You also mentioned that you have to start something with a purpose. I think you also touch on it at the beginning, whether you choose microservice or monolith, but it has to have a purpose. So tell us more about this angle. What do you mean by having a purpose, and choosing the best architecture and technology based on that?

[00:22:21] Vaughn Vernon: Sure. So, let me start from this angle. What we actually try to emphasize early in the book is to take a monolithic approach to the software development from the standpoint of keep your code together instead of distributing it on the network. Because the network will introduce all kinds of problems that will interrupt and disrupt your progress. Because you’re going to start solving problems that are not business problems. So again, we’ll get into this, well, let’s deploy these five brand new microservices, and all of a sudden, we’re no longer solving business problems anymore because we’re going to have problems. So now we have to go find these new technology mechanisms that will make this easier, will make us more successful, faster, and so forth. And look, you haven’t even solved a business problem yet. Because you just got interrupted by the technology problems you are facing.

So instead, change that angle. Focus first on just delivering software that works. When you get a sort of cadence, maybe a term that I should or shouldn’t use, but the point is where you develop as a team, a rhythm for delivering software on a steady basis with increasing value, that will start to define your success, and it will give you a lot of confidence. So we’re really saying, develop the software in modules first. Use modularity. Then you can deploy early in a monolithic way, and it keeps the network largely out of the equation. And once you get that kind of rhythm established and you’re regularly delivering value, and you’re seeing pleasure from your early users, and they’re like, wow, this is good. And then you just keep delivering on that. Hey, that’s going to be very good for a lot of people. And I’ll give you an example of that right now.

So, my team, VLINGO startup, that we have a reactive platform, XOOM, and we’re also creating this new product called DomoRoboto, which is based on, at least initially, EventStorming. So we have an EventStorming tool that also once you’ve done your EventStorming, can transition automatically into a context map. So you get the sense of the bounded context through some early ideas in the EventStorming, and then, that will display as bounded context, and now you can set up the integration relationships between them. And then, you can even look at what we call the Topo architecture, which is not topology, but topography, as in elevations and tech contours and textures, like you would see in topography map kind of presentation. So you can visualize the interactions between multiple bounded contexts with incoming events or incoming commands and outgoing events and internally, what are the entities that handle these incoming requests or the translations from events. So now, each of those areas will eventually be the ones that I described, and the other tools that are on the way will eventually be separated from one another. But right now, they’re being modularised.

Another important point is a really hard problem to solve is the remote collaboration of multiple modelers, who are now working in EventStorming. We are not trying to solve that problem right now. Actually, we have it working. But we’re not trying to deliver that out to teams right now, because we know that that will interrupt our progress on the actual value that we want to add through the primary tooling. And then, once we have a well-established set of tools, we will introduce the remote collaboration. You can go there now and check it out. It’s DomoRoboto, but it’s DomoRobo.to. Okay. And we’ll give you a link to that, but you can actually register for free and see how that works. But the point is, we are focusing on the business value. The innovation around the tooling, and then we will introduce the network later. So we’re kind of like eating our own dog food from the standpoint of we’re following what we advise other teams to do.

[00:26:54] Henry Suryawirawan: I’ll make sure to put that in the show notes. So I like the way that you explained, you start with the innovations, the value that you want to do. You have that rhythm of cadence, the experimentation feedback loops, that kind of thing, and then you start with the modules. So once you have this concept of innovations, you want to start with modules, maybe start with the monolith per se, right? So remove the equation of the distributed network and all these complexities communication between services, and you choose the deployment techniques that are more complicated later on, once you have seen some kind of values or maybe some kind of patterns.

[00:27:24] Conway’s Law

[00:27:24] Henry Suryawirawan: And these days, people talk about Conway’s Law. As the guidelines for team to start breaking apart those bounded contexts, the one that you mentioned. Start from there and separate them based on this Conway’s Law. Tell us more the relationship between Conway’s Law and bounded contexts and microservices.

[00:27:41] Vaughn Vernon: Well, I have to say that every year, that longer that I have developed software, Conway’s Law has just taken on a whole new set of meanings. It just keeps gaining value in my mind. To think that Mel Conway came up with these tremendously wise insights in 1967, and nobody even recognized it as having value until 1968, when his thesis or paper was finally released into public. And then, it took Fred Brooks another four years or so to say, Hey, this has some really amazing things associated with the concepts, and he talked about it in “The Mythical Man-Month”. As regularly as I can, I sort of shout out to Mel, and tell everybody just how amazing this is. Because it stuns me that okay, languages have changed, our approach to software development, waterfall to agile, to capital A Agile to all these things have changed, but Conway’s Law is there.

I talked about it. I compare it to the law of gravity, where it is such a constant and there’s nothing we can do to get better. People talking about getting better at Conway’s Law. Like we don’t get better at the law of gravity. As I say, if we fall about 12 feet or maybe 3 meters, we could get very badly hurt or even die, and that is just a law. So what we get better at is just our realization that if I do this thing physically, I could maybe end up having the law of gravity impact my body in some very negative way. And so, we simply don’t do that, or we find another way to do that, that is safer. So the same thing goes with Conway’s Law. We don’t get better at Conway’s Law. What we do is we find ways to succeed in the face of Conway’s Law, just like we can be successful in the face of gravity.

And what Conway’s Law is, greatly oversimplified, because people say, okay, your organization structure will have an impact on your software structure. Because your organization structure determines the communication within your organization, and that communication will have a direct reflection into the software. And so, they’ll say something like, so if you have three teams working on a system, there will be three subsystems. That’s a gross oversimplification of Conway’s Law. Actually, if you have three teams, you might have seven or eight subsystems because each team could own multiple subsystems. Especially if we’re talking about bounded contexts in DDD. But, what really is the point of this is literally the people on a team may not communicate very well or the right people are not on the team. It’s like the domain expert who nobody can figure out who that should be. That’s a big problem. Because organization structure, hierarchy, whatever it is, reporting channels can just derail the effort to communicate in the way that we need to as software teams.

And then came this idea of the reverse Conway maneuver. But actually, Mel Conway gave us the solution in 1967. He said, very often the initial system that we create is not the best one. So we have to recognize that our communication has led to an inferior system product. Therefore, we should be flexible and reorganize according to the better communication structures based on what we need to design. So the reverse Conway maneuver is what I prefer to actually call Conway’s first axiom. Because he gave us the answer in the paper. He didn’t just leave us with a problem and say, okay, let’s watch everybody fail for the next 50 years, while I sit back and laugh at them because I told them so. No, he actually told us so to the extent where he said, be flexible. Reward flexibility. So not to disrespect to those who came up with this idea of the reverse Conway maneuver, but I think people just fail to read beyond that one simple statement. I think it’s just one sentence, for organizations that design systems, in a broad sense. It doesn’t mean just software systems, any kind of system. For organizations that design systems, their designs will reflect the communication structure of that organization. And they just stop there. Read the rest of the paper, read the whole paper. It’s amazing. It’s an amazing paper.

Mel Conway was saying in 1967 what people are just now figuring out, and it’s been there all the time. I’m sorry, maybe I’m just overdoing this, but the man is just amazingly brilliant. I don’t know if he’s in some kind of computing hall of fame or whatever, but he should definitely be one of the first ones in there. I mean, Donald Knuth is amazing, but truly how many people really have read all whatever number of volumes that he has and really applies it. Okay. So again, all due credit to all these great engineers and so forth, but Mel Conway is right up there with the amazing ones.

[00:33:06] Henry Suryawirawan: Thanks for the insightful explanation on this. This is my first time also seeing it in the way you explain it. I think the way you explain Conway’s Law is the same like a law of gravity. I think it’s really insightful. It’s not like you can run away from it. It will always be there. So instead of getting better at Conway’s Law, we should find a way to adapt. You know, like maybe the way we communicate, the way we structure our teams to be better, instead of maybe trying to be better at the law itself. And I like the way that you mentioned the paper has been there, but only recently that we all seem to be crazy about it. I think I’ll put it in the show notes for people who are interested if the paper is available online so that we can all read again to the essence of that article. So thanks for mentioning that.

[00:33:48] Events First Architecture

[00:33:48] Henry Suryawirawan: The other aspects that I see from the book that is very, I would say, crucial, right? You mentioned about events first architecture. So when we talk about monolith and events first, this is sometimes it’s not interrelated somehow. Because when you write monolith, you can just do function calls all over the places. Why you emphasize a lot about events first?

[00:34:06] Vaughn Vernon: Well, I have to start from the communication angle. What I’m finding is that actually, Conway’s Law, it is pervasive. It plays into every aspect. For me, I just have to start talking about things relative to communication. So in terms of communication, what actually happens to people, for example, when they take some action. They take an action because someone else took an action. Even our bodies, the way our bodies function. We have some automatic kinds of processes that run our heartbeats, our nervous system works and we breathe. So there is some sort of these automatic things, but I think we traced many of those back, we would find that it’s an event driven system. There’s something firing back there in the background.

I think as an example we used in the book, the executives are to prepare for this board meeting and they’re supposed to contribute something to an idea around the new initiative. What happens when an email hits their inbox. It says, “go do this”. Some of those executives will probably take a look right away. Some will probably say, “Oh, too busy. I can’t believe I’m being asked to do this.” Some will say, “Oh, I resonate with this. I’m going to contribute.” And others say, “There’s someone else in my organization who will do a much better job at advising me about this.” But the point is they’re all reacting or will react in some way. Some will decline. No, I can’t do this. Some will sign up, and some may pass on, like I’m going to delegate this to someone else. But they’re reacting to it. What is the event? The event is that someone created an invitation, and gave some parameters to what the contribution to the new initiative must be, and now, the executives on the board who need to prepare for this are reacting to it. So events are very basic in the human nature and the way that we communicate. It’s sort of amazing that we’ve allowed the way computers work to influence our way of communicating. So when we are making everything like a procedure and not reacting to events, then we’re really sort of leaving out a very important part of human communication. And so, our communication structure is actually being kind of derailed by the ways computers work. So stop doing that and make computers work for us for the human communication experience rather than us work for them.

Another important point about this is that events are very really simple to reason about in software. When we see something happen, like a user, Will says, “Do this.” They’ll send a command, “do this”. But the “do this” when it’s done, it ends up being an event whether or not we service that event explicitly or not, there is still an event that happens. We can go all the way back to some very early pioneering thought in the early 1900s about this, where it was recognized that events were natural things in human life, in communication. Even up to today, there’s a book called “Geometry of Meaning”. And the “Geometry of Meaning” shows that events are very basic in everyday life in just communication. So I think it’s very natural. The more I’ve thought about it, the more consideration I’ve given it, it just seems like the really natural thing to do.

And what people don’t realize is that events were there as early as the initial graphical user interface. Every single operation gesture that the user makes that causes the software to acknowledge or to be informed that the user did something is an event. As early as the Model View Controller in Smalltalk, and possibly even before Model View Controller. But this led to some really interesting concepts of now, the computer or the software is communicating with us by saying, “this happened”. So to me, it’s just quite natural, and people become afraid of event driven. And yet, most everybody has programmed in some kind of GUI, even if it’s just a browser GUI. Not like this Windows API or X Windows mind-blowingly large API. But the simple API, like in the browser with the DOM. I mean, this is event-driven architecture. And so if you can understand that UI construct, you can understand event-driven architecture and you can understand reactive. It’s all right there, and you’ve been using it all along. So don’t scare yourself. You already know this. It’s just, maybe some words you’re not familiar with.

[00:38:54] Embrace Latency

[00:38:54] Henry Suryawirawan: The way you explain it makes sense, like, for example, the way we behave in life also based on events, right? Maybe we just don’t abstract it that way, but you can see it all over the places. Events being triggered, events being processed, and events being created. And there’s also one important aspect of event-driven as well, you also cover it in the previous episode, which is about latency. The sequence of events happening, they are not happening in one atomic way. So normally, they will be some delays or some people call it asynchronous or maybe eventual consistency. But latency is there. The time aspect is there. Maybe you can cover it a little bit more about this latency part.

[00:39:29] Vaughn Vernon: Sure. Well, let me say this. I want to step back again and just use another kind of illustration. If you go to an online dictionary and you type in the word “reactive”. The basic meaning behind reactive sounds negative. Because it’s usually talking about humans being reactive, as opposed to proactive. So go to the same dictionary or look up somewhere in encyclopedia, look up proactive and look up reactive. And it sounds like proactive is way better than reactive. But it’s semantics, actually. Proactive individuals are still reactive individuals. They are being proactive because something that they were made aware of, so they are reacting to an event or a set of events. If you want to frame human personalities or tendencies that proactive is a better way of operating as a human than being reactive, because maybe reaction gives negative connotation of I’m emotionally charged by something and I’m going to just lash out at it or whatever. That’s not how I’m thinking of reactive. Being proactive is really just a matter of being reactive, but doing so early. Taking earlier responsibility, kind of like those executives who said, “Oh yeah, I can contribute to this. This is really in my zone, so I can contribute to this initiative and I’m going to do that.” That’s more of the proactive, whereas the delegation is sort of like pushing it down the road. But neither one of those are negative. It’s just simply a fact that some are more prepared to deal with that kind of event than others are.

So the latency part of it. Now, when you think about the fact that these events get surfaced and someone or some software component will react to them in time, it actually takes a lot of pressure off of the architectural design. I don’t actually remember in detail what I spoke about a year ago regarding the latency and so forth. But I basically call this embracing latency. This is where we are saying, okay, we’re doing a lot of work that involves a network. Wires across machines, satellites, whatever it happens to be, that is part of the communication. And sometimes, things because of the law of physics, just simply cannot operate at the same speed as we would like them to. Or that we think they will because when we were developing the software, “Oh, isn’t it always true that when we develop the software, it just reacts? The timing is just awesome.” But if we try to do that same test at three or four in the morning when backups are being run on the network, you would find a different story. And I can testify to that because I have spent plenty of three and four AMs trying to get things to work and being impacted by database backups or whatever across the network. So when a network is saturated, there’s really nothing you can do about it.

Now, what does that mean? It means that instead of saying, I’m going to make this REST request, and if I don’t get a response to that request within five seconds, we’re timing out. That’s sort of what we have to plan for because we don’t want that thread disappearing for 15 seconds, 30 seconds. So we kind of have no other choice than to just cut off the communication because it’s taking longer than we think it should or longer than we can afford for it to take. Part of that is just, well, look, other requests are stacking up in our WebSocket channels, and we need to process the next thing or we’re going to crash maybe from overly impacted by new requests, by not handling the current requests expediently. So we just take that whole mindset and we shifted and we say, “Let’s send a message, and whenever we get a message back, we’ll do something about it.”

So we decouple the temporal aspect that we were counting on before. We were saying, this must happen within five seconds. And we think because when we test, it usually happens within a second or less. Hey, awesome. Five seconds should be plenty. And at certain times, five seconds isn’t plenty, and we’re going to fail certain operations. And especially if that operation is a compensating action from a previous transaction. Ooh. Big problems. So instead, what we do is we publish events, or we put communication in an asynchronous channel. Somebody finds out about that in time. It’s sort of a guarantee that will happen, and there’s sort of a guarantee that we will find out later whether that succeeded or not. Or at least we can set up a timer that says if it exceeds this amount of time, like maybe five seconds, is not a real business concern. It’s just simply a technology decision that we’re going to fail this REST request after five seconds. We’re just going to close the channel and move on. Whereas the business might say, actually, we could wait an hour for that to happen. It’s really not important. Oh. But we have to have it in no more than five seconds. Otherwise, you should see what happens to our front end. That’s just the wrong way to think about it. Let’s let the business decide if this could take an hour or if this could take 15 minutes. Whatever it is, it’s probably going to be far more than five seconds.

A lot of times that this kind of data harmony can occur on another subsystem. Then we introduce that and now we’re embracing latency. We’re saying we’re acknowledging that the network exists. We’re acknowledging that the business has a say in it, not simply an arbitrary technical decision. We involve the business. Literally, what happens if your company, your business, is trying to solve problems for a very good customer, client, and we fail things because of a technical limitation, and that happens regularly to that customer, client. Are they going to be happy? No. That is not the kind of decision that we want like a software developer with a few years of just software development experience and maybe very little business experience making and just saying, “Yeah, five seconds is plenty.” When in reality, we should be going to the business and saying, “Well, we’ve got this situation. Here’s what’s happening. If we blocked a thread and a communication channel for more than five seconds, we start to see problems on the front end.” Or maybe it’s even the business saying to us, “You must have the entire use case should happen within no more than five seconds.” And so we say, “Yeah, what we’re seeing is that this subsystem that we need to interact with, collaborate with is slower than that on a lot of occasions. Could we maybe lighten that timeout?” “Well, yes, we could do that.” But then someone else on the team says, “Yeah, but you can’t increase the timeout because if we do, then we’re just going to have more requests stacking up on the front end. It’s just going to escalate into potential failures on the front end.” So we still have to do something about this. Well, we can still take the fact that the business says maybe 15 seconds is fine or maybe 30 seconds or even a number of minutes. And then we can take that from the business and make decisions in the software design that will satisfy the customers, the clients that are paying us to help them to have better outcomes, not worse outcomes.

[00:47:12] Henry Suryawirawan: So I like the way that you mentioned about acknowledging network, acknowledging a distributed system, and acknowledging business has a say in this decision as well. So embracing latency is not just technical decision, but it’s also related to the business aspects and also the architecture of the network and the distributed system.

[00:47:30] Monoliths vs Microservices

[00:47:30] Henry Suryawirawan: So Vaughn, as we go towards almost the end of the interview, maybe to just give a gist of the book in summary about monolith versus microservice. What is the message that you want to convey for the audience here?

[00:47:41] Vaughn Vernon: Well, the subtitle of the book is “Driving innovation, using purposeful architectures”. So the emphasis is first on strategy. Strategic and then monoliths and microservices, and then driving innovation, “comma”, so to speak, using purposeful architecture. So we’re taking the monolith and microservice, and attaching that to purposeful architecture, but emphasizing the strategy and driving innovation. So those are kind of two separate thoughts. But what we’re saying is that use the strategic and innovative drivers to help us determine what does our architecture needs to be. Make it have purpose. And by the way, since so many companies have made mistakes with both monoliths and microservices, here is sort of a formula that is going to help you to have a much more successful architecture software design. Because even though we don’t consider this a DDD book, we do include four chapters, kind of a lightweight introductory chapters on domain-driven design, and what’s the problems it solves and how it helps us.

But then also the purposeful architecture we have that second part of the book is on domain-driven success. The third part of the book is on events first architecture. We’re helping people understand what is message driven? What is event-driven? And why does it help us succeed? We actually can even succeed with event-driven and message driven using REST, even asynchronously, and we get into that. But then also, what are the patterns around process managers or sagas and being orchestration or when should we use choreography? What is choreography versus orchestration?

And then the final fourth part of the book is on the two approaches to purposeful architecture. At least the ones that we’re sharing in this book are monoliths like you mean it. In other words, we didn’t just accidentally come up with this monolith because nobody had any better ideas at the time, or didn’t recognize how bad it was getting. So monolith like you mean it and microservices like a boss. It’s actually been a year since I’ve written those chapters and it might amaze you, at least this author doesn’t go back and reread his books a lot. So maybe my memory of that is a little bit fuzzy right at this point.

But the point is we’re trying to help people understand. You’re messed up. Hey, lot of people mess up. Don’t feel so bad, but correct it. Now, we’re going to help you correct it. Here’s how you can correct that monolith, and here’s how you can even correct that monolith to the point where you can move the modules in the cleaned up monolith to microservices. Maybe not all of them. Maybe you don’t need to move all of them into a separate deployment unit, but maybe some of them you do. Or maybe for some reason you want to use multiple monoliths. So it’s really just trying to help people take a mature look at the real important things in software. And then support the efforts to follow through with those really important things of software by using architecture that will help you be successful.

And so our final chapter, I think, is really well-named. I mean, I thought a lot about this name and, of course, Tomaz had a lot to do with the book as well, but I can remember thinking this title through and it’s something like “require balance and demand strategy”. So the point is we got to be balanced in software. The patterns that we use, the mechanisms that we use, and don’t just use these as new toys for your own pleasure, but to actually solve problems. Because the demand should be on strategy. Demand it of your software developers. You’re paying a lot of money for these very intelligent and talented people. Use them for your purpose and make your company better. Be a leader in your industry or industry segment. So that’s the real message.

[00:51:54] Henry Suryawirawan: I do like that phrase. So require balance, demand strategy, right? So always comes with a purpose, not arbitrary decisions. And monolith is not always bad. I think I’ve also covered it in a few episodes. So all agree that monolith is not only bad things happening, but big ball of mud monolith, which you mentioned in the book is bad. So don’t do that. Please modularise that.

[00:52:16] 3 Tech Lead Wisdom

[00:52:16] Henry Suryawirawan: So, Vaughn, it’s been a pleasure having this conversation. It’s really insightful. I learned a lot. I asked you this question last time, which is the three technical leadership wisdom. So I’m not sure whether you remember what you said last time. I just want to give it another try this time, or whether you come up with the different versions of the three technical leadership wisdom.

[00:52:34] Vaughn Vernon: Well, I probably did say something about communication. But no, truly, I don’t remember exactly what I said. But I think it was formulated from the conversation itself. The things we had discussed. So I’m going to say communication leads to innovation, first. So communicate well. Man, honor Mel Conway’s legacy. And this is a great legacy, right? Call it heritage. We’ve inherited his wisdom. Use that knowledge and fix your communication structures. Learn how to communicate. And software developers, come out of your shell. Learn how to communicate with business people. Help them trust you instead of dislike you. So communication is extremely important.

When you do communicate, make the communication count. And make wise software decisions. Model well. Go beyond the obvious. Treat every opportunity to learn more about the business or the aspect of the business that you’re working in. Take in all the knowledge you can. Not so that you can become a domain expert. Because you probably will never be a domain expert, but you will understand that mental model, what is referred to as the image schemas that are in other people’s minds about how the business should work, and then model that in software.

And then, just be mature. I think that the other two aspects are summarized in just be mature. Look, if you can write some kind of great algorithm that you learned from Donald Knuth, excellent. That’s awesome. But don’t go looking for places to use it. Have enough knowledge in your head about all the options that you have, that when you have a purpose for those, you know which option to choose. That’s maturity.

[00:54:31] Henry Suryawirawan: Thanks for reiterating your three technical leadership wisdom. I still remember the first one related to communication as well, but you mentioned it such that programmers need to go out of their cubicles. So it’s almost similar.

[00:54:41] Vaughn Vernon: Well, very similar. Yeah.

[00:54:43] Henry Suryawirawan: Thanks again, Vaughn, for this time. Where can people find the book? Where they can find you online?

[00:54:48] Vaughn Vernon: Yeah. Well, by the time you’re hearing this podcast, hopefully, all of the various sales channels are open. Amazon in the US, Amazon in Europe and so forth. Whatever your general European or Asian distributions are, they should be all working smoothly by, I’m guessing maybe January of 22. Everything should be working smoothly. Right at the moment, supply chain with COVID has just been very difficult to deal with and predict how things will go. But I think by then, by the time the podcast is out, you should be able to go to your regular favorite book acquisition site or bookstore or something and find it. And we would appreciate a favorable review on Amazon or Goodreads or any other places that other people are going to learn from. So thank you for reading, if you do.

[00:55:37] Henry Suryawirawan: So, yeah, in preparation of this interview, I read the book. Not in full, of course. It takes time to read it. But I must say that it is really an insightful book. It covers end-to-end from the business aspects of it, the architecture part, and also the last question that we always ask, “Is it monolith or microservice?” So do find the answer in the book and I wish you to have a very good insightful read out of the book. Thanks Vaughn for writing that. Thank you so much for attending this call. I really enjoyed it.

[00:56:03] Vaughn Vernon: I did as well. Thank you so much, Henry. Again, my pleasure and honor to be with you again. And I look forward to maybe our next conversation. So, take care.

[00:56:12] Henry Suryawirawan: I know that you have another book planned, so we’ll do that next time.

[00:56:15] Vaughn Vernon: Okay. Cool.

– End –