★ Event store optimizations in laravel-event-sourcing

About a year ago, we released laravel-event-projector. It focused on adding projectors, an important concept in event sourcing, to Laravel.

After the release of the package, we continually kept improving it. We added aggregates, a way to test those, a brand new section in the our documentation that explains event sourcing from scratch, and DX improvements all across the board.

We now feel confident that the package is a good starting point for getting started with event sourcing in Laravel. That's why we're renaming the package to laravel-event-sourcing.

The package also includes some memory optimizations, which were added by my colleague Rias. In this blog post, I'd like to walk you through the changes.

Using your own event store

In previous versions laravel-event-projector, the storage of events was tied to Eloquent. We had a StoredEvent model that handled the (de-)serialization of an event.

Before an aggregate can be used, it needs to be reconstituted from previous events. If you have a very active aggregate, that needs to be rebuilt using a great many events, the time serializing and deserializing StoredEvent models will add up.

That's why we added an interface that allows you to use any storage you want. This would allow you to create an event store in which manually build queries (for example with DB:: statements, without using Eloquent).

namespace Spatie\EventSourcing;

use Illuminate\Support\LazyCollection;

interface StoredEventRepository
{
    public function retrieveAll(string $uuid = null): LazyCollection;

    public function retrieveAllStartingFrom(int $startingFrom, string $uuid = null): LazyCollection;

    public function persist(ShouldBeStored $event, string $uuid = null): StoredEvent;

    public function persistMany(array $events, string $uuid = null): LazyCollection;

    public function update(StoredEvent $storedEvent): StoredEvent;
}

If you need an example, you can take a look at how we implemented EloquentStoredEventRepository, which is the repository that will be used by default.

After creating your implementation, don't forget to specify the class name of your repository in the stored_event_repository key of the config file.

Memory optimizations

You might have noticed the usage of LazyCollection in the StoredEventRepository. This is a feature introduced in Laravel 6. Under the hood, it leverages PHP's generators to work with large collections, while still keeping memory use low.

Let's take a look at the implementation of retrieveAll in EloquentStoredEventRepository

public function retrieveAll(string $uuid = null): LazyCollection
{
    /** @var \Illuminate\Database\Query\Builder $query */
    $query = $this->storedEventModel::query();

    if ($uuid) {
        $query->uuid($uuid);
    }

    return $query->orderBy('id')->cursor()->map(function (EloquentStoredEvent $storedEvent) {
        return $storedEvent->toStoredEvent();
    });
}

Notice the usage of cursor() here. This will significantly reduce memory usage because Laravel will, at any given moment, only hold one model in memory. Under the hood, this uses MySQL cursors. As I understand it, instead of having the results in memory of the laravel app, MySQL will hold a result set and send the results one by one to the Laravel app.

Rias performed some benchmarks using his Macbook Pro (processor: 2.3 Ghz Intel Core i5). These are the numbers for calling retrieveAll with 100 000 events to be retrieved.

laravel-event-projector with Collection:

• peak memory usage: 267MB of memory
• execution time: 7.0077209472656 seconds

laravel-event-sourcing with LazyCollection:

• peak memory usage: 18MB of memory
• execution time: 7.478010892868 seconds

Obviously, peak memory usage is much lower/better using LazyCollection at the expense of a slightly worse execution time.

Upgrading from laravel-event-projector the package

If you were on v3 of the laravel-event-projector, upgrading to laravel-event-sourcing is easy. You have to perform these steps:

1. Change "laravel-event-projector":"^3.0" to "laravel-event-sourcing":"^v1.0" and run composer update
2. The namespace has changed, so you need to replace Spatie\EventProjector by Spatie\EventSourcing in your entire project.

If you're still on v1 or v2 of laravel-event-projector then you need to upgrade to v3 first like described in our upgrade guide.

In closing

I'm proud of how our package grew from a simple way to handle projectors to the easiest way to get started with event sourcing in Laravel. I did not do this by myself. My colleagues Seb & Rias did a lot of polishing, I got some good advice from my buddy Dries. Frank De Jonge's EventSauce package inspired some of the features.

laravel-event-sourcing isn't the first package that was built by Spatie. Take a look at this big list of Laravel and PHP stuff we've opensourced.