Initially, we have done immediate consistency across the board. It is easiest to reason about and create initially. And I would say if it is an internal business app, it probably never needs to scale, and therefore could stop there.
But even on a single node, full consistency some negative implications.
One is because of the coupling between events and view, deployments affect everything. When I deploy a new view, I risk also taking down the command API and vice versa.
Another is that other services interested in listening to events can become really awkward to implement. Take an emailer (that gets throttled by SES). To keep it consistent, we had to implement a fully-consistent projector to record the fact that emails need to be sent (based on events that got generated from the domain). Then we have a timed service that polls the query API every minute to see if there are emails to send. Then it calls the command side to actually send the emails and record they were sent. It's a really awkward workflow for the developer because of full consistency.
When we move over to eventual, the service can listen to events as they come in (instead of polling every minute), update it's own view of what it is doing, and save events indicating emails were sent. Far more straight-forward.
Another is scalability. Our load is more read-heavy. With full consistency, we would need to engage the view database's replica feature to add read capacity. This involves going deeper with (and being restricted by capabilities of) the projection DB -- settings, replication lag issues, etc. But with eventual, we don't even need to go that far. We can literally just spin up a new copy of the projector service, a new independent database for it, and add the service to a load balancer. We can also scale it down easily by doing the inverse. And basically every listener service works the same way (only with different integrations) so it is already something we know.
The really hard part of eventual consistency is accepting limitations around set validations and memberships. I've been meaning to write a post on that.
Lead Developer and Solutions Architect, I specialise in Event Sourcing, DDD and Event Driven systems. PHP and GoLang developer. Enjoys being a smart ass and having a nice whiskey.
Location
Ireland
Education
MSc in Computer Science, Trinity College, Dublin
Work
Lead Developer and Solutions Architect at Contractor
I've experienced the exact same issues you mention above, and once we made the switch to eventual we realised how easy it made things.
The funny thing is, you can architect your system as eventually consistent, then just force it to run as immediately consistent. Once you run into problems, switch over to eventual. It allows you to defer the problem, and it makes it easy to change when you need to. Building it as a immediate, then switching to eventual is a much more costly change, mainly because the immediate implementation is probably not event driven.
We used immediate for domain projections (mostly for set validations and memberships, as you phrased it) because we need them to guarantee domain constraints. There are other solutions though (think I discussed some in a previous article) but I'd love to see someone (maybe yourself :)) explore it further.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Awesome as always.
Initially, we have done immediate consistency across the board. It is easiest to reason about and create initially. And I would say if it is an internal business app, it probably never needs to scale, and therefore could stop there.
But even on a single node, full consistency some negative implications.
One is because of the coupling between events and view, deployments affect everything. When I deploy a new view, I risk also taking down the command API and vice versa.
Another is that other services interested in listening to events can become really awkward to implement. Take an emailer (that gets throttled by SES). To keep it consistent, we had to implement a fully-consistent projector to record the fact that emails need to be sent (based on events that got generated from the domain). Then we have a timed service that polls the query API every minute to see if there are emails to send. Then it calls the command side to actually send the emails and record they were sent. It's a really awkward workflow for the developer because of full consistency.
When we move over to eventual, the service can listen to events as they come in (instead of polling every minute), update it's own view of what it is doing, and save events indicating emails were sent. Far more straight-forward.
Another is scalability. Our load is more read-heavy. With full consistency, we would need to engage the view database's replica feature to add read capacity. This involves going deeper with (and being restricted by capabilities of) the projection DB -- settings, replication lag issues, etc. But with eventual, we don't even need to go that far. We can literally just spin up a new copy of the projector service, a new independent database for it, and add the service to a load balancer. We can also scale it down easily by doing the inverse. And basically every listener service works the same way (only with different integrations) so it is already something we know.
The really hard part of eventual consistency is accepting limitations around set validations and memberships. I've been meaning to write a post on that.
Thanks Kasey.
I've experienced the exact same issues you mention above, and once we made the switch to eventual we realised how easy it made things.
The funny thing is, you can architect your system as eventually consistent, then just force it to run as immediately consistent. Once you run into problems, switch over to eventual. It allows you to defer the problem, and it makes it easy to change when you need to. Building it as a immediate, then switching to eventual is a much more costly change, mainly because the immediate implementation is probably not event driven.
We used immediate for domain projections (mostly for set validations and memberships, as you phrased it) because we need them to guarantee domain constraints. There are other solutions though (think I discussed some in a previous article) but I'd love to see someone (maybe yourself :)) explore it further.