Passionate developer in Java and Scala. And sometimes, something else. A few months per year, someone calls me "professor". CoFounder of Scala By The Lagoon @scalagoon
I am curious to know how many requests/s and events/s the system was handling.
For the time I was operating it, a few hundreds/minute at peak. The main focus was multitenancy, and there was a queue of clients to be migrated on the new system.
The presentation has pretty valuable information for others looking to use Kafka as an event store. Other event storage strategies will face slightly different challenges.
Kafka is one of the few "distributed logs" on the market. The critical feature is the durability of the log, since it may happen that it needs to be replayed. You can't do it with a pure messaging platform because they usually are not optimized for long-term durability. A database (relational or not) must have specific features to manage append-style tables quickly to be suitable for this load. Cassandra is a popular choice (and one of the default production event store reccommended for Akka Persistent Actors).
How did you handle snapshot structural changes?
With care.
Your changes must be as back-compatible as they can, and the write side must be ready to be very liberal in what it reads. If your domain changes wildly, you have to manage this change and assign it to non-structural fields; that is, prepare to have lots of Map<String, String> and such. If you use them with the appropriate care, you can go a long way without needing to change the event or snapshot schema. A bit of analysis up-front can save you lots of problems in the operation of the system.
changes to business logic which require tracking different data in the write-side snapshot
I think the only way to work this is to re-read history. Or call it a different system and restart from 0. If the new data needs the old events, you have little choice.
or writing code to migrate snapshots to new structure.
Snapshots are so short-lived that you usually consider only one, the last one. So, your code (or, to be more precise, your unmarshalling layer) must be able to read just the last one. Each change you make, you can remove the code to read version n-2, because there will not be any snapshots around in that format.
Thanks for sharing! This is a lot of valuable information.
We have an event sourced system in production and are developing another one. We use different strategies and techs, so our trade-offs are a bit different. It was very interesting to see how your choices played out in trade-offs. I very much appreciate you sharing.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Here is the complete link:
slideshare.net/michele.mauro/5-thi...
For the time I was operating it, a few hundreds/minute at peak. The main focus was multitenancy, and there was a queue of clients to be migrated on the new system.
Kafka is one of the few "distributed logs" on the market. The critical feature is the durability of the log, since it may happen that it needs to be replayed. You can't do it with a pure messaging platform because they usually are not optimized for long-term durability. A database (relational or not) must have specific features to manage append-style tables quickly to be suitable for this load. Cassandra is a popular choice (and one of the default production event store reccommended for Akka Persistent Actors).
With care.
Your changes must be as back-compatible as they can, and the write side must be ready to be very liberal in what it reads. If your domain changes wildly, you have to manage this change and assign it to non-structural fields; that is, prepare to have lots of
Map<String, String>and such. If you use them with the appropriate care, you can go a long way without needing to change the event or snapshot schema. A bit of analysis up-front can save you lots of problems in the operation of the system.I think the only way to work this is to re-read history. Or call it a different system and restart from 0. If the new data needs the old events, you have little choice.
Snapshots are so short-lived that you usually consider only one, the last one. So, your code (or, to be more precise, your unmarshalling layer) must be able to read just the last one. Each change you make, you can remove the code to read version n-2, because there will not be any snapshots around in that format.
Thanks for sharing! This is a lot of valuable information.
We have an event sourced system in production and are developing another one. We use different strategies and techs, so our trade-offs are a bit different. It was very interesting to see how your choices played out in trade-offs. I very much appreciate you sharing.