DEV Community

loading...
Cover image for Leveraging DynamoDB Split for Heat: an Architecture Suggestion

Leveraging DynamoDB Split for Heat: an Architecture Suggestion

ndrestone profile image André Updated on ・2 min read

Disclaimer: I'm not affiliated with AWS and this is not technical advice. In case you consider using this pattern in production, please be aware the core feature is very briefly documented and there is actually no performance guarantee.

According to the docs, DynamoDB's auto-split, split-for-heat or Adaptive Capacity auto-split is a (briefly documented) feature that "rebalances your partitions such that frequently accessed items don't reside on the same partition".

After performing some tests to have a better understanding on how this feature actually works, and to acknowledge that it isn't a replacement for write sharding, I came up with this architecture as a way to improve read costs for, maybe, some use cases.

Lazy Sharding

Lazy Sharding

Lazy Sharding

Basically, the idea here is to only write-shard the partition if our requests are being throttled. If they are, run a de-sharding strategy to rewrite the items to a single partition, while DynamoDB takes care of the repartitioning. When reading, also perform an attemptive read on the main partition first, only performing a sharded read as a second attempt.

Alternatively, for cases where the reads are disproportionally high, not allowing the partitioning to happen based on write velocities, similar de-sharding jobs can be run, so attemptive reads on a single partition can be done in order to save RCUs.

Is it Worth it? Would it work?

Honestly, in addition to the fact that there's no performance guarantee on the underlying feature, I'm not sure if there are many use cases for this. So, please tell me what you think, leave your feedback!

--
André

Discussion

pic
Editor guide