Introduction to Event Sourcing and CQRS

In a galaxy far, far away, enter the saga of CQRS and event sourcing, where data updates unfold like an epic space opera. Jokes away, let's see what are those

Event sourcing is not a new term. Event Sourcing is a powerful paradigm for managing data in the ever-evolving software development landscape. While it might not be as widely known as other data management methods, Event Sourcing offers a unique approach that can benefit applications significantly.

Command Query Responsibility Segregation (CQRS) is an architectural paradigm that divides the responsibilities of read and write operations within a system.

At its heart is the notion that you can use a different model to update information than the model you use to read information Martin Fowler

In the realm of CQRS application architecture, the system is separated into two distinct segments. The first caters to the realms of updates and deletions – a territory we call the writing model. Concurrently, the second segment takes on the noble task of reading – aptly named the read model. Unlike the conventional CRUD approach, which relies on a single database, CQRS embraces two databases (or at least different tables). Each side is obsessed with the act of reading or writing.

To address scalability concerns, CQRS can be coupled with event sourcing. In this scenario, events generated by commands are stored in an event store. These events are subsequently asynchronously transmitted to a separate read data store, undergoing a transformation process to align with the read data model. This integration helps overcome some of the scalability challenges inherent in CQRS.

In this article, we will delve into the fundamentals of Event Sourcing coupled with CQRS, explore its advantages, and discuss when it might be better to avoid using this approach.

What is event-sourcing

If you work with technology, you must have come across it. It's a puissant tool used by many large organizations for data modeling. It can scale and meet the needs of the modern data processing industry.

Event sourcing is a compelling architectural pattern that might initially seem a bit eccentric. Instead of focusing on the state of your system, it keeps track of every change that happens. It's like holding a detailed diary of every emotional rollercoaster it goes through. All the events that are changing the state of your system are recorded, and such records serve as both a source for the current state and an audit trail of what has happened in the application over its lifetime.

Domain experts usually describe their systems as a collection of entities, which are containers for storing a state and events, representing changes to entities as a result of processing input data within various business processes. Events are often triggered by commands invocated by users, background processes, or integrations with external systems. The naming is usually defined by Ubiquitous Language.

Ubiquitous Language is a domain-driven design (DDD) concept, a software development approach. Imagine Ubiquitous Language as the magical Babelfish of software development. It's the secret code that lets developers, domain experts, and stakeholders all speak the same lingo, like a universal translator for geeks and business folks. With this shared language, the jargon barrier becomes a thing of the past, and everyone can boogie on the same wavelength.

Among existing practices to define the ubiquitous language, I'd emphasize Event-storming. However, sometimes it may not be the best option (team or resource constraints, limited initial knowledge, etc). Then, the process may be neglected, and developers apply the most widely used terms.

Many architectural patterns treat entities as a primary concept. These patterns describe how to store them, how to access them, and how to modify them. Within this architectural style, events are often "on the side": they are the consequences of entity changes. Unlike the traditional systems, with an event-sourcing approach, the events are considered the only source of truth.

At first glance, it may sound unusual, but most of the serious systems we know and interact with do not emphasize the concept of the current state or the final state (financial, banking, and many others). As Greg Young (creator of CQRS) said in one of his speeches, your bank account is not just a column in a table but the sum of all transactions that occurred with your bank account (renewals, write-offs, and recovery). For example, if you have a disagreement with your bank, the balance sheet and the bank confirm that the balance is 69 dollars, but your position is still 96. You won't hear a response like "The column states it's 69 dollars, and it's all you have".

Designing systems with a focus on events and event logs provides the following benefits:

• It helps reduce impedance mismatches and the need for concept mapping, allowing technology teams to "speak the same language" (ubiquitous language) as the business when discussing the system.
• Encourages separation of responsibility into commands and queries (command/query responsibility segregation), allowing you to optimize writing and reading independently of each other.
• It provides temporality and a history of change, allowing questions to be answered about what the system looked like at specific points in the past and what events occurred before that point.

Key Concepts of Event Sourcing

Events

Events are the fundamental building blocks of Event Sourcing. They represent discrete changes in the state of an entity. Each event is immutable and contains information about what happened when it occurred and any relevant data associated with the change.

You can register as many events as you like. The name of the event should have clear semantics. Events are always talking about the past and can reveal with their names what has changed and how it has changed (for example, OrderCanceled, OrderItemAdded, ProductRemoved, OrderPlaced).

Event Store

The Event Store is the central repository for storing events in the order they occurred. It ensures that events are appended to the end of the log and provides methods to read, write, and query events.

Aggregates

Imagine Aggregates as a consistency boundary around a group of domain objects, such as entities and value objects. In Event Sourcing, Aggregates are reconstituted from a single fine-grained event stream (e.g., representing an order flow). During this operation, the current state of the aggregate is calculated so that it can be used to handle a command.

The state is a crucial part of an Aggregate toolkit. It's like their memory is wiped clean after every command unless they're into some fancy snapshotting business. Consider it a superhero's utility belt, equipped to deal with duplicate commands and other unexpected villains, thanks to at-least-once delivery guarantees. It's not playing movies; it's making decisions.

Projection

Projections are used to derive the current state of an entity or a view of the data by replaying events. Projections are separate from the Event Store and can be optimized for specific query needs.

An integral part of Event Sourcing is the concept of snapshots – intermediate state captures of an aggregate in the EventStore. Sometimes, to obtain the final state of an object, you need to replay many events, starting from the very first one. To optimize this process, snapshots are taken, and we will restore the final state not from the very first event in the system but from the latest snapshot. Despite the apparent benefit of this pattern, it should only be applied when obtaining the final state takes excessive time. Many might think that a downside is the exponential growth of data volume with this approach, but that's not the case. Events are storing not the complete data models but only the state changes. In reality, the slowdown in such a system is associated with the construction of aggregates from all domain events that have occurred until now. So, Event Sourcing is the storage of a series of events, and the data schema that reflects these events is a direct derivative of these events. The data schema in Event Sourcing systems is temporary and can be rebuilt or reconstructed from events at any time. Isn't it like a time machine?

Benefits of Event Sourcing: The Fun Side of Managing Data

Event sourcing coupled with CQRS promotes decentralized modification and reading of data. This architecture scales well and is suitable for systems that already work with event processing or want to migrate to such an architecture.

1. Complete Audit Trail: With Event Sourcing, you've got a complete record of your software's shenanigans. It's the ultimate way to check if your software's been up to no good or just having a few harmless adventures.
2. Temporal Querying: It's like going back in time to see what your software was thinking and doing at a specific moment. Wondering why it behaved oddly on a Tuesday six months ago? Event Sourcing has your back.
3. Parallel Processing: Event-sourcing turns your software into a multitasking genius. It can handle multiple tasks simultaneously, like a magician juggling flaming torches. Events are processed asynchronously, so your software can easily handle high loads.
4. Distributed Systems: In a distributed system, events can be processed asynchronously and independently, leading to improved scalability. Each component or microservice can consume and process events at its own pace without blocking others.
5. Flexibility: Event-sourcing is like a chameleon for data – it can adapt to different situations. It allows you to create various "disguises" for your data, tailored to specific needs. You could achieve the same with other approaches (like Postgres materialized views or building your own reports). However, event-sourcing allows you to achieve the same in a very natural way.
6. Fault Tolerance: Your software's history is safe and sound. In case of mishaps, you can simply turn back the clock and replay events to restore order in your software universe.

When to Avoid Event Sourcing: Not Every Party Needs a Diary

As much as we'd love to make Event Sourcing the life of the party, it's not always the best guest for every occasion. Here's when it's better to opt for something else:

1. Simplicity: If your application is as uncomplicated as a one-page novel, Event Sourcing might be like using a sledgehammer to crack a walnut. It's great for epic tales but overkill for short stories.
2. Performance: Event Sourcing might seem too leisurely for applications that demand real-time updates. Due to eventual consistency, the data may be slightly outmoded
3. Overhead: In read-focused systems, traditional databases may be more suited for your "need-it-now" attitude (building the projections would require some extra computing power)
4. Learning Curve: Implementing Event Sourcing can be like learning a new language. It's not something you can master overnight, so if your project is on a tight schedule or your team is new to this, be prepared for a learning adventure.

In short, shorter, shortest…

Event Sourcing is like the eccentric uncle who turns up quirky but surprisingly insightful at family gatherings. It offers a unique approach to managing data, providing complete auditability, temporal querying, scalability, flexibility, and fault tolerance. However, like any eccentric guest, it might not be the best fit for every occasion. So, choose Event Sourcing when the story is epic and complex, but feel free to opt for traditional methods when your tale is short and sweet. After all, the software world is diverse, and there's room for both diary keepers and straightforward note-takers. In the upcoming article, I'll share practical hints on implementing event-sourcing and CQRS in the Rails world.