#66 - Time and Temporal Modeling in Event Sourcing - Tomasz Jaskula
“Time is important for business. We have to model it explicitly. Temporal modeling means that we use time-based artifacts as first modeling citizens.”
Tomasz Jaskula is the CTO and co-founder of Luteceo and an experienced software developer and architect. In this episode, we started off discussing how Domain-Driven Design (DDD) influenced Tomasz’s view on software development approach and its relation with functional programming. Tomasz then explained in depth about the time concept in business applications and temporal modeling, in particular, bi-temporal modeling. He mentioned the different concepts of time in temporal modeling, explaining them using an example for easier illustration. We then extended our discussion further to Event Sourcing, understanding the key concept, its relation to temporal modeling, when we should decide to use Event Sourcing in our application, and some available tools that can help us implement Event Sourcing.
Listen out for:
- Career Journey - [00:04:58]
- DDD and Bounded Context - [00:08:56]
- DDD and Functional Programming - [00:13:24]
- Temporal Modeling - [00:14:47]
- 3 Different Types of Time - [00:21:13]
- Event Sourcing - [00:25:42]
- When to Use Event Sourcing - [00:28:13]
- Event Sourcing Tools - [00:34:02]
- 3 Tech Lead Wisdom - [00:36:10]
_____
Tomasz’s Bio
Tomasz Jaskuła is CTO and co-founder of Luteceo, a software consulting company in Paris. Tomasz has more than 20 years of professional experience as a developer and software architect, and worked for many companies in the e-commerce, industry, insurance, and financial fields. He has mainly focused on creating software that delivers true business value, aligns with strategic business initiatives, and provides solutions with clearly identifiable competitive advantages. Tomasz is also a main contributor to the OSS project XOOM for the .NET platform. In his free time, Tomasz perfects his guitar playing and spends time with his family. He recently wrote a book with Vaughn Vernon titled “Strategic Monoliths and Microservices” published by Addison-Wesley.
Follow Tomasz:
- LinkedIn – https://www.linkedin.com/in/tomasz-jaskula-16b2823/
- Twitter – @tjaskula
- Luteceo – http://luteceo.com
Mentions & Links:
- 📚 Strategic Monoliths and Microservices: Driving Innovation Using Purposeful Architecture – https://amzn.to/3hkwWBx
- 📚 Domain-Driven Design: Tackling Complexity in the Heart of Software – https://amzn.to/3smbS3L
- 📚 Implementing Domain-Driven Design – https://amzn.to/3IrRaFr
- XOOM – https://vlingo.io/
- .NET XOOM – https://github.com/vlingo-net
- Event sourcing – https://docs.microsoft.com/en-us/azure/architecture/patterns/event-sourcing
- EventStorming – https://www.eventstorming.com/
- Command and Query Responsibility Segregation (CQRS) – https://docs.microsoft.com/en-us/azure/architecture/patterns/cqrs
- Vaughn Vernon – https://vaughnvernon.com/
- Alberto Brandolini – https://www.linkedin.com/in/brando/
- VB6 – https://winworldpc.com/product/microsoft-visual-bas/60
- .NET Framework – https://en.wikipedia.org/wiki/.NET_Framework
- EventStoreDB – https://www.eventstore.com/eventstoredb
- Allen’s Interval Algebra – https://www.ics.uci.edu/~alspaugh/cls/shr/allen.html
Tech Lead Journal now offers you some swags that you can purchase online. These swags are printed on-demand based on your preference, and will be delivered safely to you all over the world where shipping is available.
Check out all the cool swags available by visiting techleadjournal.dev/shop. And don't forget to brag yourself once you receive any of those swags.
DDD and Bounded Context
-
The important thing in DDD is the strategic part. At the very beginning, around 2007-2010, people were focused more on the tactical side of DDD, patterns like repository, entities, value objects. I’m not saying they are not important. Of course, they are important artifacts when it comes to DDD implementation, but the whole strategic part when you build ubiquitous language with business people and other members of your team, and you are discovering business problems from the strategic point of view. So that’s what makes DDD worthful.
-
Identifying different contexts of the model is one of the parts of DDD, which is called bounded context. Bounded context says that this term of this concept does really have a specific meaning in this specific context, and may have another meaning in a different context.
-
Once you’ve identified those contexts, you have to find a way of communicating and integrating together. So that’s also another part of strategic Domain Driven Design that is very important.
-
You can go along with implementation and tactical tools from Domain-Driven Design, but you cannot just pick one or the other. You have to use the strategic and tactical part all together.
DDD and Functional Programming
-
In functional programming, you have these concepts of immutability. And in DDD, you have the concept of domain events. Domain events are facts that have occurred, and you cannot change them. You can just acknowledge that something has happened.
-
The fact that you have this unit that is important part of the domain modeling matches well with functional programming. It’s maybe more visible when you are using something like Event Sourcing and CQRS architectural styles, when you have this command which is the intention and you have the function that will take the command in, apply the state and producing new state.
Temporal Modeling
-
All non-trivial businesses and business problems are time related.
-
Coming back to business, most of the time I’ve seen that models that we were working on don’t take time as the first ingredient, and the time is always implicit, like everything would happen instantly.
-
There are artifacts like workflows, business processes that are time-based. Those are hints to us that time passes and time is important for business. So what temporal modeling means is that we use time-based artifacts as first modeling citizens.
-
As well, domain events, they are related to time and passing of time. The important thing to notice is that business always thinks including time, so we have to model it explicitly. If we don’t model time as an explicit thing, this is where many problems creep in. We try to solve things, bringing in accidental complexity, which leads in a long run to unmaintainable software and big ball of mud.
-
There are different types of temporal modeling. There can be triggers that are produced by passing time.
-
Another artifact I was mentioning is our domain events. A domain event is an immutable fact. Something that the business people care about. It is presented as a verb in a past tense. As this is a fact, we cannot change it. It’s something that has happened, and we can just acknowledge. But we cannot change it. It’s immutable.
-
Using domain events in your model when you’re modeling and creating a domain model is an indication of time as well. Because by publishing an event, you say to everyone else that is interested in this kind of domain event, that something has happened, and it’s up to different interested parties listening to those domain events to act upon them. But at the same time, it doesn’t say it has to happen right after. It introduces some kind of asynchronous relationship, and you don’t really know if someone will act upon like in a millisecond, in a second, or in three days.
-
This is like an indication of time, and it’s defined with the business people. Most of the time when you don’t take into account the time, developers or people implementing the domain model are trying to happen everything instantaneously at the same time. And it creates problems because we have to create different kinds of transactional models, so everything is happening atomically. But it’s not the case when it comes to temporal modeling.
3 Different Types of Time
-
There are different types of temporal modeling, and all make the time explicit in domain modeling. Those different types are bi-temporal, tri-temporal, and I know if we can go further, and they all to answer different questions.
-
Bi-temporal modeling is based on two time axes.
-
As you said, there is a transactional time, valid time axis. Because of that, we can answer specific questions like, what the state of my system was at a certain point in time, or what the state of my system should have been at a certain point in time, and to understand what has happened.
-
The idea behind it is to use the transaction time as the time when the transaction has happened. For example, if we record something in the system, we will take the current date, and put it on the transaction time. But the valid time is defined freely.
-
-
Tri-temporal modeling case goes even further because you have a third time axis, which is the decision axis. It makes queries even more complex, but it gives you as well more power in the way you would like to explore facts.
Event Sourcing
-
The traditional way to persist state is to save the current state. Event sourcing uses a radically different event-centric approach to persistence. Business object is persisted by storing a sequence of state changing events.
-
So whenever an object state changes, a new event is appended to the sequence of already produced events, and in the append-only store. That’s how the current state of the business object is reconstructed, by replaying all the events that have happened through the object lifecycle lifetime.
-
Event sourcing is different because before you identify the most important fact, events that are important to the business and to your model, that every time something changes, you produce a chunk of the state. So an event that will be appended to the log, to the event source log, which is append-only log.
-
That way, when you look at what kind of events were appended to the event log, you will understand what has happened. Exactly how we arrive, how we get to the current state. Because all the events can tell you what was happening to get to the current state.
-
You cannot do that in a traditional approach when you are just discarding the previous state and overwriting it with the new snapshot of the state.
When to Use Event Sourcing
-
Event sourcing allows you to look at your domain from different points of view. Because not only you know how you got the current state, so this gives you knowledge about what is happening in your system or in your domain. You never know how you could use those events in the future. So it gives you even more data about your domain that you can use in different ways in the future. So that’s an important information that you get for free when using event sourcing, that you don’t get when you’re using the traditional approach.
-
State transitions are a very important part of a problem space, and it should be modeled as is, and event sourcing helps with that. So we can see through different events that have happened. The different state changes, and to really know what have happened. So that’s very important information. I would say, if you have a complex business domain, and you were working with domain events, using event sourcing is easier than the traditional approach.
-
I’ve been using event sourcing almost every time on refactoring old legacy projects. So what we were doing is to synchronize legacy system with the new event source contexts through events. Even though your legacy system is not event sourced, you just have the snapshots of the current state. There are several techniques you can use to synchronize legacy system with the new event source, let’s say, microservice.
-
The easiest one is to produce events on the legacy side. It can happen either by using tools like Change Data Capture that we scan all the transactions that happen, the data sources in legacy systems. They can produce meaningful events that you can listen in your new context, and migrate the old state from legacy to the new context.
-
Or you can produce those events yourself with business people. You can identify what is important. If there are changes that happen on legacy side, you can produce events that are meaningful for the new context, and you can synchronize in that way and have event source context and new context or microservice that is used that way.
-
Or it happened on some of the project I was working on, that the legacy system had some historical data that was stored in like historical tables. We could just derive events and what has happened from the history tables. You can use it in greenfield and brownfield without problems.
-
It’s not easy. Sometimes you don’t have access to all the legacy parts that take part in state changes. So, there will be some information that is missing because it’s not always one-to-one.
-
But most of the problems, if you can just produce some meaningful events on the legacy side, and then synchronize them to the new context, you will be able to link those two different systems, legacy and new context.
Event Sourcing Tools
-
There are event source databases. But you can use as well, SQL database, which has this ability. Or you can even store events in the tables in SQL relational database using some libraries that help you out with it. So there is no problem with storing events.
-
After what you will choose, it depends on the constraints you have. Because you can have performance problems. You have like millions of events that happen in a minute, so you need something reliable and sometimes storing it in SQL database won’t cut it.
-
When it comes to bi-temporal, this is where the things get more complex, because the best way would be to have a storage that is bi-temporal compatible. Most important thing is that you could run different queries on different time axis. So definitely, the database or storage we use have to comply with the standards related to temporal databases, because otherwise it will be very hard to do it.
-
There is, in fact, the standard for bitemporal databases, which is called SQL 2011.
3 Tech Lead Wisdom
-
Don’t be afraid to get out of your comfort zone, learning and trying new things, opens mind to see how you would solve differently problems you used to solve in the past.
-
Experimentation.
-
In software development, I couldn’t see many teams doing that enough, and experimentation is often seen as a waste of time by the stakeholders and people that are financing projects.
-
But I think it’s an important source of knowledge and software is about knowledge. I mean, it’s even more about knowledge than technology.
-
Experimentation definitely opens new possibilities of acquiring this knowledge.
-
-
Communication is a very important part of software development, even more important than the technology itself.
- Often where the software goes wrong is because this part is neglected. If you don’t focus enough on communications, on the dynamics between different teams and how teams are organized, you will experience very hard the Conway’s Law, which in fact says that the software that is produced, in a sort, matches the patterns of communication and the organization.
[00:01:12] Episode Introduction
[00:01:12] Henry Suryawirawan: Hello again to all of you my friends and listeners. Welcome to another new episode of the Tech Lead Journal podcast. Thanks for being here with me today, listening to this episode. If you’re new to Tech Lead Journal, I invite you to subscribe and follow the show on your podcast app and our growing social media communities on LinkedIn, Twitter, and Instagram. And if you have been regularly listening and enjoying this podcast and love the type of content that I’m producing, will you support the show by subscribing as a patron at techleadjournal.dev/patron, and support my journey to produce great episode every week.
My guest for today’s episode is Tomasz Jaskula. Tomasz is the CTO and co-founder of Luteceo and an experienced software developer and architect. He’s also a main contributor to the OSS project XOOM, X-O-O-M, for the .NET platform. He recently wrote a book with Vaughn Vernon titled “Strategic Monoliths and Microservices” published by Addison-Wesley.
In this episode, we started off discussing how Domain-Driven Design influenced Tomasz’s view on software development approach and how DDD relates with functional programming, especially when dealing with domain events. Tomasz then explained in depth about the time concept in business applications and temporal modeling, in particular bitemporal modeling. He mentioned the different concepts of time in temporal modeling, explaining those different types using an example for easier illustration. We then extended our discussion further to Event Sourcing, understanding the key concept and its relation to temporal modeling. Tomasz also gave some practical tips on when we should decide to use Event Sourcing in our application, and some available tools that can help us to implement Event Sourcing.
I enjoyed my conversation with Tomasz, learning about temporary modeling, the different types of time, and it felt like being in an Inception movie moving along the different time dimensions. And if you also enjoy this episode, please share it to someone you know who would also benefit from it. You can also leave a rating and review on your podcast app or share your comments on the social media on what you enjoy from this episode. I always love reading your reviews and comments, and they are one of the best ways to help me spread this podcast to more people. And it is my hope that they can also benefit from this podcast. So let’s get our episode started right after our sponsor message.
[00:04:13] Introduction
[00:04:13] Henry Suryawirawan: Hello, everyone. Welcome back to a new episode of the Tech Lead Journal podcast. Today I have with me a guest named Tomasz Jaskula. He’s French. So Tomasz is actually a very experienced software developer and architect. He has a career over 20 years. So that’s really a long time for me. He’s also the main contributor of open source project called XOOM for the .NET platform. His specialty is Domain-Driven Design, functional programming. And also, later on we’ll be talking a lot about temporal modeling and event sourcing that kind of topics. So Tomasz, thank you so much for attending this conversation. So looking forward to have a chat with you and learn all these concepts.
[00:04:51] Tomasz Jaskula: Thank you Henry for having me in Tech Lead Journal. It’s a great pleasure to talk about all those topics important to me.
[00:04:58] Career Journey
[00:04:58] Henry Suryawirawan: So Tomasz, in the beginning, maybe you can introduce yourself for those people who may not be aware of you yet. Maybe specifying your highlights or turning points in your career.
[00:05:08] Tomasz Jaskula: Okay. So yeah, let’s start maybe from the beginning. So I graduated in 2001 from a French university, and like many people, I started to work as a developer for some consulting company. At that time, I was mostly passionate about technology, and I didn’t really understand the deeper meaning of software development, or in other words, that the software development is not just about technology and technical skills. The funny thing is that, if I remember well, everyone was thinking that we could just solve any complex business problem with technology. That time, it was VB6 and .NET framework, which I work with till now, was at its inception. There was no Stack Overflow to copy paste from. So yeah, you can imagine. I mean, some developers couldn’t just do their job today. Joke aside. Then in 2007, I was so confident in my skills that I founded my own company with friends I graduated with. And I’ve learned the hard way that running a company requires a lot of different skills. So that was the hard part of the journey. Since then, I’ve been always working either as a freelancer or founding other companies. So that was a big turning point in my career.
But really what changed my way of looking at software development approach was the discovery of Domain-Driven Design. I discovered Eric’s book around 2007. At the time, I was struggling to put into practice everything I’ve learned with more or less success. There was not really good literature out there because you know, Eric’s book was more high level than it might seem the first sight. Generally, people were struggling with different use cases, and when Vaughn Vernon published his red book, “Implementing Domain-Driven Design”, I could really see different strategies and trade-offs with Domain-Driven Design implementation.
So, another turning point was when I met Vaughn in person around 2013. It was during the “Implementing Domain-Driven Design” tour in Belgium that was organized when his book was published. So there, I met all the prominent members of today’s DDD community. We were all learning from Vaughn at the time. And the funny thing is that at that time I discovered as well, Alberto Brandolini’s EventStorming technique for discovery based on domain events that is a part of Domain-Driven Design. This is so popular now, but they had a chance to be one of the early adopters of EventStorming before it became so popular. The truth is that I’ve been learning all the time. There were many things I’ve learned that would be what much need here, but we won’t spend too much time on that.
Maybe another highlight is when I learned functional programming. It was a real paradigm shift in thinking on how to solve business problems that I used to solve using mutations and Object Oriented Programming. The great thing about functional programming is that, in fact, it matches very well designing with Domain-Driven Design approach, and specially domain events. So that was something I found really useful. So as you said, I’ve been programming since 20 years and still I’m programming. I run my own company, software contributions. And the last achievement I’m proud of is to be co-author with Vaughn Vernon of the book, “Strategic Monoliths and Microservices” that will be published by the end of December.
[00:08:56] DDD and Bounded Context
[00:08:56] Henry Suryawirawan: Thanks for sharing your story. It’s really interesting how you evolve in your career. I read Eric Evans’ books as well, like a few years ago, like long time back. It was a thick book. Very dry, just to be honest. I personally struggled when I read the book. And actually, DDD becomes one of the maybe buzzwords or trends in the technology industry today. Well, when you say that you were actually learning DDD, and get these turning points in your career, can you probably summarize? I know it’s probably tough. What exactly that captivates you with DDD? And how that actually transform your thinking or your career?
[00:09:30] Tomasz Jaskula: Yeah. So the important thing in DDD is the strategic part. At the very beginning, I mean around 2007-2010, people were focused more on the tactical side of DDD, you know, patterns like repository, and entities, value objects. I’m not saying they are not important. Of course, they are important artifacts when it comes to DDD implementation, but the whole strategic part when you build ubiquitous language with business people and other members of your team, and you are discovering business problems from the strategic point of view. So that’s what makes DDD worthful. Every business problem is complex enough. So you have some sort of model that will be useful to solve the current business task. Identifying different contexts of the model is one of the parts of DDD, which is called bounded context.
Bounded context says that this term of this concept does really have a specific meaning in this specific context, and may have another meaning in a different context. So, that was a real discovery for me, because before that, I was trying to stuff everything in one model, and try the model that was trying to solve too many different problems. Let’s give an example, maybe for easier understanding. Let’s say we are in e-commerce domain. We have a product that can be sold on the e-commerce page. So if we speak about those different contexts, we might have like catalog context, shipping context, billing context. We can find the product concept in all those contexts, but they have different meanings. If you are in a catalog context, we care about description of the product. How it will be seen by people that are browsing products catalog. But if we are in the shipping context, we will more think about rules and properties that are specific to shipping. We really care more about the way and how different shipping rules apply in that context. So, you know, you have those different strategic parts that you have to discover when using Domain-Driven Design and that’s I think what was the most important thing for me to discover and helped me with many projects, by the way.
[00:12:04] Henry Suryawirawan: Yeah, I think bounded context is such an important concept in DDD. I also learn myself personally as well, that it’s the same ubiquitous language, like in your example is product. But actually it could mean something different in different contexts. And people translate context these days to microservice, as in terms of the good scope of a microservice.
[00:12:23] Tomasz Jaskula: Yes, of course. And there is more, because once you’ve identified those contexts, you have to find a way of communicating and integrating together. So that’s also another part of strategic Domain Driven Design that is very important. I mean, you can go along with implementation and tactical tools from Domain-Driven Design, but you cannot just pick one or the other. You have to use the strategic and tactical part all together. At the beginning of Domain-Driven Design, we were like focusing more on tactical part than on strategic part. Because there was not enough literature out there and people doing Domain-Driven Design. So implementation of the integration between different context was most important thing at the time. But now many books were published since then. And I think the most important is the red book by Vaughn. So now everyone is Domain-Driven Design expert.
[00:13:24] DDD and Functional Programming
[00:13:24] Henry Suryawirawan: Yeah. I can see that topics all over the places, actually. Another thing that I picked up from your introduction just now. You mentioned that you learn functional programming and you actually think that functional programming matches well with DDD. So when I learned about DDD, a lot of concepts, actually kind of like OOP, Object-Oriented programming. So can you explain a little bit here? What do you mean by matching well with functional programming?
[00:13:47] Tomasz Jaskula: Yes. In functional programming, you have these concepts of immutability. And in DDD, you have the concept of domain events. Domain events are fact that has occurred, and you cannot change them. You can just acknowledge that something has happened. So the fact that you have this unit that is important part of the domain modeling matches well with functional programming. It’s maybe more visible when you are using something like Event Sourcing and CQRS architectural styles, when you have this command which is the intention and you have the function that will take the command in, apply the state and producing new state. So, you can like make the relation between functions and immutability of functional programming, and how this can be applied to Domain-Driven Design, and especially using Event Sourcing and CQRS architectural styles.
[00:14:47] Temporal Modeling
[00:14:47] Henry Suryawirawan: So speaking about domain events and event sourcing and things like that, it’s a good segue actually, to move into our main topic today, which is about temporal modeling. So you have been dealing a lot with systems that have this kind of concept. So maybe in the beginning, can you explain to all of us here, who probably are new to this concept, what is actually the definition of temporal modeling?
[00:15:10] Tomasz Jaskula: Yeah. All non-trivial businesses and business problems are time related. Let’s take an example. I don’t know if you know much about music and composition. But everyone I think has seen how music is written. You have the notes, the stuff, and you’re just writing the music on the stuff using notes and different musical notations. This is like a model of presentation of the music. You can share this model with different musicians around the world, and if they know how to read, they will be able to play the music from the model you’ve shared with them. The important thing is that the model represents not only the structure, because each note is like a pitch of a note, so you know which sound it will produce. But as well, the time it’s encoded. The time is the value of the notes. So if you didn’t encode the time into the music notation, no one would be able to make sense of what has to be played. Another hint is that not only time is encoded, but you have a concept called tempo, which says how fast the time is for the music. So all those three things make it possible for the people to play music that is shared through the model, like the stuff and music notations.
Coming back to business, most of the time I’ve seen that models that we were working on don’t take time as the first ingredient, and the time is always implicit, like everything would happen instantly. There are artifacts like workflows, business processes that are time-based. Those are hints for us that time passes and time is important for business. So what temporal modeling means is that we use time-based artifacts as first modeling citizens. I said workflow, business processes, but as well, domain events, they are related to time and passing of time. The important thing to notice is that business always thinks including time. So we have to model it explicitly. If we don’t model time as an explicit thing, this is where many problems creep in. We try to solve things, bringing in accidental complexity, which leads in a long run to unmaintainable software and big ball of mud as well, which is a very big problem in the software industry.
[00:17:45] Henry Suryawirawan: Yeah. Maybe if you can illustrate, what do you mean by having time as the first class concept in your business problem? Because I think all of us probably are so used to building business solutions, but actually we probably are not aware how to actually make the time as the actual first class concept.
[00:18:03] Tomasz Jaskula: Yeah. So there are different types of temporal modeling. Let’s say, we can take as example, a business process that can be implemented in our architecture as some kind of process manager. So this is like time-based. There can be triggers that are produced by passing time. Let’s say, we have business process which will allow to expire offers that were sent to customers, but customer didn’t use the discount code or something like that. You could have this process that is running, and by the time defined by the business, let’s say, the offer was valid for 30 days. When 30 days pass, there is a time trigger which may be implemented. There’s domain event, like offer expired that will trigger the process of the offer expiration. So we can define the business rule saying, “send another offer to the customer or we just close the current offer”, and the different kinds of actions business can define here based on the time passing.
Another artifact I was mentioning is our domain events. Let’s start with the domain event. A domain event is a immutable fact. Something that the business people care about. It is presented as a verb in a past tense. Account opened. Money deposited. Money withdrawn. They are actions that are important to the business and they’re modeled as domain events. So as this is a fact, we cannot change it. It’s something that has happened, and we can just acknowledge. But we cannot change it. It’s immutable. Using domain events in your model when you’re modeling and creating a domain model is an indication of time as well. Because by publishing an event, you say to everyone else that is interested in this kind of domain event, that something has happened, and it’s up to different interested parties listening to those domain events to act upon them. But in the same time, it doesn’t say it has to happen right after. It introduces some kind of asynchronous relationship, and you don’t really know if someone will act upon like in a millisecond, in a second, or in three days.
So this is like an indication of time, and it’s defined with the business people. Most of the time when you don’t take into account the time, developers or people implementing the domain model are trying to happen everything instantaneously in the same time. And it creates problems because we have to create different kinds of transactional models, so everything is happening atomically. But it’s not the case when it comes to temporal modeling. So as I said, business processes, domain events, workflows, they all are implementations artifacts that you can use to model this time in your business model.
[00:21:13] 3 Different Types of Time
[00:21:13] Henry Suryawirawan: When I researched this topic, there are three different levels that I can see about this temporal modeling with the three different concepts of time, or the time types, so to speak. They are called valid time, transaction time, and decision time. Maybe you can explain what are all these different time concepts?
[00:21:32] Tomasz Jaskula: Yeah. So as you mentioned, there are different types of temporal modeling, and all make the time explicit in domain modeling. Those different types are bitemporal, tri-temporal, and I know if we can go further, but already bitemporal and tri-temporal a bit complex, and they all to answer different questions. For example, bitemporal modeling is based on two time axes. As you said, there is a transactional time, valid time axis. Because of that, we can answer specific questions like, what the state of my system was at certain point in time? Or what the state of my system should have been at certain point of time? And to understand what has happened.
So, this might be a bit confusing how those two axes work together. But the idea behind it is to use the transaction time as the time when the transaction has happened. For example, if we record something in the system, we will take the current date, and put it on the transaction time. But the valid time is defined freely. So, let’s take an example. Let’s say you’re a clerk in town hall and you are recording when a child is born. I know it’s no longer happening like that, but let’s just pretend. And you said, my child was born on that date, so please record it, and the clerk is doing so. So, bitemporal, how it would be used with this specific case. Let’s say, someone is coming in on 5th of April, and says my child was born on the 3rd of April. So what would the clerk do is he would record that fact. A child is born, and the transaction time would be the 5th, because it’s the day he’s doing the operation, and he would set the valid time to 3rd of April. So that way, you can answer different question, because if your viewpoint is at 4th of April, and you are using only transaction time, you wouldn’t see any child born on 3rd of April. But if you use the axis of valid time, you’ll see that a child was born on 3rd of April. So that’s a very simple use case, even though it’s very hard to explain. But this is what bitemporal modeling allows. And tri-temporal modeling case goes even further because you have a third time axis, which is the decision axis, and that’s very difficult to make those queries over three time axes.
[00:24:13] Henry Suryawirawan: If you can explain a little bit on this decision time, so maybe extending from the same example of the child is born. What will be an example of the decision time for tri-modeling?
[00:24:24] Tomasz Jaskula: Yeah, the decision time is set as well by the clerk. So the fact that the child was born on the 3rd of April, that would be the valid time. But in that case, the decision time, I know from the business point of view, I don’t think it has any meaning here, it probably would be set to the day of today, because decision was done by the clerk on that day as well. So maybe the example, if I was to extend, someone comes in to the town hall and says, I would like to register my son born two days before, so on 3rd of April. And the clerk would just take note of it on a piece of paper. It would pass to another service that will record the fact in the system. If the recording happens on 7th of April, that transaction time would be the 7th April. The decision time would be the 5th because this is the decision that was made by the first clerk to record the fact into the system. The valid time would be the 3rd of April because this is the time the child was born. I would say that way. It makes queries even more complex, but it gives you as well more power of the way you would like to explore facts.
[00:25:42] Event Sourcing
[00:25:42] Henry Suryawirawan: So I think I can personally even struggle to understand this different time. It’s like Inception movie, I guess, you have different times. It’s multi-layered. But I think I would probably focus it a little bit to the discussion of bitemporal modeling, where people these days are familiar probably with the concept of event sourcing. Event sourcing is also something that comes quite related with DDD. So maybe if you can explain fundamentally also what is actually event sourcing?
[00:26:07] Tomasz Jaskula: Yes. So the traditional way to persist a state is to save the current state. Event sourcing uses radically different event centric approach to persistence. Business object is persisted by storing a sequence of state changing events. So whenever an object state changes, a new event is appended to the sequence of already produced events, and in the append-only store. That’s how the current state of the business object is reconstructed is by replaying all the events that have happened through the object lifecycle lifetime. So maybe give a little example, let’s say, of the shopping cart. You could just like store the current state of what items do you have in your shopping cart, and what’s the total amount of it if you have coupon code or not. Every time when user is appending new item or is changing something, we just discard the previous state and we record the new snapshot of this state.
So event sourcing is different because before you identify the most important fact, events that are important to the business and to your model, in a way, that every time something changes, for example, an item was added to a shopping cart, you produce a chunk of the state. So an event that will be appended to the log, to the event source log, which is append-only log. That way, when you look at what kind of events were appended to the event log, you will understand what has happened. Exactly how we arrive, how we get to the current state. Because all the events can tell you what was happening to get to the current state. And you cannot do that in a traditional approach when you are just discarding the previous state and overwriting it with the new snapshot of the state. So that’s how it differs.
[00:28:13] When to Use Event Sourcing
[00:28:13] Henry Suryawirawan: So some people actually also illustrate this concept with the version control system. It’s also append-only mostly, like Git version control system where all the changes that you made is actually recorded as a diff of the change. And then you can replay, move back in time, what revision, or even find a specific snapshot in time. And I mean, when I conceptualize that to business problems, sometimes I think it’s a bit complicated. I haven’t personally built an event sourcing system before. What do you think will be some of the typical business applications for event sourcing? Is this something that is for all business problems? Or is it for only specific class of problems?
[00:28:50] Tomasz Jaskula: Yes. What I’d like to say about event sourcing is that really, it allows you to look at your domain from different point of view. Because not only you know how you got the current state, so this gives you knowledge about what is happening in your system or in your domain. You never know how you could use those events in the future. So it gives you even more data about your domain that you can use in different ways in the future. So that’s an important information that you get for free when using event sourcing, that you don’t get when you’re using the traditional approach.
Event sourcing is not a very new idea. In fact, it was used around seventies or something like that. So it’s a very old idea. For example, all the relational databases, like SQL Server and others are using this concept of event sourcing to actually internally store transactions. So that’s very interesting. In businesses, it’s been like 10 years that we started to look at event sourcing more closely. Because one of the drawbacks that we can say, it is that the storage takes more volume in terms of data. Now it seems that it’s not a problem. We can store almost any data as we want. In the early seventies, eighties, nineties, it was not the case. So that’s why people used relational databases at first. What’s important to notice is that state transitions are very important part of a problem space, and it should be modeled as is, and event sourcing helps with that. So we can see through different events that has happened. The different state changes, and to really know what have happened. So that’s very important information. I would say, if you have a complex business domain, and you were working with domain events, using event sourcing is easier than the traditional approach, in my opinion.
[00:30:58] Henry Suryawirawan: And is this something that can only be used for greenfield projects? Or actually when you have a legacy system, business complex applications that is legacy, can you introduce this event sourcing concept? Can it be mixed? What do you think about this?
[00:31:13] Tomasz Jaskula: Yes. I’ve been using event sourcing almost every time on refactoring old legacy projects. So what we were doing is to synchronize legacy system with the new event source contexts through events. Even though your legacy system is not event sourced, you just have the snapshots of the current state. There are several techniques you can use to synchronize legacy system with the new event source, let’s say, microservice. So obviously, the easiest one is to produce events on the legacy side. It can happen either by using tools like Change Data Capture that we scan all the transactions that happen, you know, the data sources in legacy systems. They can produce meaningful events that you can listen in your new context, and migrate the old state from legacy to the new context. Or you can produce those events yourself with business people. You can identify what is important. If there are changes that happen on legacy side, you can produce events that are meaningful for the new context, and you can synchronize in that way and have event source context and new context or microservice that is used that way. Or it happened on some of the project I was working on, that the legacy system had some historical data that was stored in like historical tables. We could just derive events and what has happened from the history tables. So yeah, you can use it in greenfield and brownfield without problems.
[00:32:56] Henry Suryawirawan: So I think what I can learn from your explanation is that we have to still focus back on the domain events or the events itself, right? Like even though you have a legacy, you can start producing events. Maybe at the derivatives of that, there’s another system that stores and captures that events in an event sourcing way. And then maybe from then on, yeah, you can start having this concept of produce me the state of the system at this time. I guess if I understand the things correctly?
[00:33:20] Tomasz Jaskula: Yeah. yes. This is how it works. I mean, it’s not easy. Sometimes you don’t have access to all the legacy parts that take part in state changes. So, there will be some informations that are missing because it’s not always one-to-one, and hopefully you would like to have different kinds of modeling event source or the newer context using event sourcing. But most of the problems, if you can just produce some meaningful events on the legacy side, and then synchronize them to the new context, you will be able to link those two different systems legacy and new context, like your microservices or something else.
[00:34:02] Event Sourcing Tools
[00:34:02] Henry Suryawirawan: You mentioned just now that traditionally or historically that it has been a challenge to implement this kind of system because of the relational database concept. Are there any up-and-coming tools specifically built to solve this kind of event sourcing problem?
[00:34:16] Tomasz Jaskula: Yeah. There are tools that you can use. There are event source databases. It’s called, I think, EventStoreDB. But you can use as well, SQL database, which has this ability. Or you can even store events in the tables in SQL relational database using some libraries that help you out with it. So there is no problem with storing events. After what you will choose, it depends on the constraints you have. Because you can have performance problems. You have like millions of events that happen in a minute, so you need something reliable and sometimes storing it in SQL database won’t cut it. So it depends on what you need, but storing events, it’s not a problem. But when it comes to bitemporal, this is where the things get more complex, because the best way would be to have a storage that is bitemporal compatible. Most important thing is that you could run different queries on different time axis. So definitely, the database or storage we use have to comply with the standards related to temporal databases, because otherwise it will be very hard to do it.
There is in fact, the standard for bitemporal databases, which is called SQL 2011. It’s kind of temporal databases standard which deals with automatic time periods, splitting temporal primary keys, functional integrity, and interval, Allen’s interval algebra, and time slate, sequence queries. So this is all important properties that you are using when you are doing queries over temporal models. That’s why it’s important to have those kinds of databases.
[00:36:10] 3 Tech Lead Wisdom
[00:36:10] Henry Suryawirawan: Thanks for sharing these tools. So speaking about time, it’s very interesting and time has passed, actually. So we are actually reaching the end of our recording here. But before I let you go, normally I ask this particular question to all my guests, which is called the 3 technical leadership wisdom. So maybe Tomasz, would you be able to share your version of 3 technical leadership wisdom for us?
[00:36:31] Tomasz Jaskula: What’s important for me is to not be afraid to get out of your comfort zone, learning and trying new things, opens mind to see how you would solve differently problems you used to solve in the past. That’s one thing I could share.
Another one is experimentation. In software development, I couldn’t see many teams doing that enough, and experimentation is often seen as a waste of time by the stakeholders and people that are financing, maybe projects. But I think it’s an important source of knowledge, and software is about knowledge. I mean, it’s even more about knowledge than technology. Experimentation definitely opens new possibilities of acquiring this knowledge. So the inventor of the light bulb, Thomas Edison, for example. He had made a number of different experimentations that failed, and he said, “I have not failed. Not once. I’ve just discovered 10,000 ways that don’t work.” So this kind of knowledge is very precious as well.
The last one is that communication is very important part of software development, even more important than the technology itself. Often where the software goes wrong is because this part is neglected. If you don’t focus enough on communications, on the dynamics between different teams and how teams are organized, you will experience very hardly the Conway’s Law, which in fact says that the software that is produced, in a sort, matches the patterns of communication and the organization. That’s why it’s really important to take this into account as well. So maybe they are related one to another, but I think it’s very important to take into account.
[00:38:27] Henry Suryawirawan: Wow! Thanks. I like the last one, the communication. So when you mentioned that communication maybe potentially be the source of the many problems in software development. So yeah, I tend to agree with that one as well. So Tomasz, thanks again for sharing your knowledge. It’s really, really a pleasure to talk about time. Actually, I rarely do this. But for people who want to learn more from you, maybe they would like to explore this topic further, where they can find you online?
[00:38:52] Tomasz Jaskula: On Twitter, mostly. My Twitter handle is @tjaskula. Maybe in the description you can just leave it for the people. But yeah, mainly on Twitter.
[00:39:03] Henry Suryawirawan: Right. I’ll make sure to put that in the show notes. Thanks Tomasz for this conversation. Hope to see you again sometimes, and good luck with your book publishing.
[00:39:11] Tomasz Jaskula: Thank you for inviting me. It was a pleasure to take a video about the topics and I hope to see you soon.
– End –