DynamoDB design
Tomasz Kuczma
In previous article , I explained you how to create a service like TinyURL. That kind of service requires storing a lot of data what implies sharding this data across multiple DB instances. I showed a really simple algorithm of how data could be distributed in that case and briefly mention something about replication. Now the question is what would we do in real life?
Real-life distribtued DB
Using existing NoSQL solution would be a good choice as they already solve main problems with distributed DB systems:
- Data replication and durability
- High availability even in corner cases like adding new DB shard to the system and hardware failures
- Scalability and capacity management
- Security
Examples of such systems are AWS DynamoDB and Google Cloud Bigtable. Both are very similar but store data in a slightly different way hence give different guarantees. I would like to focus on the first one today.
DynamoDB design
Now, I would like to talk a little bit more about DynamoDB system design as this is the most interesting part for engineers. How is it build? How data is distributed and replicated? What happens in case of hardware failure?
Unfortunately, “jasso” - Senior Principal Engineer in AWS already did that on his presentation “Amazon DynamoDB Under the Hood: How We Built a Hyper-Scale Database” during re:Invent 2018.
The presentation have a ton of knowledge on how to design a mature large-scale system. Enjoy!
Software engineer with a passion. Interested in computer networks and large-scale distributed computing. He loves to optimize and simplify software on various levels of abstraction starting from memory ordering through non-blocking algorithms up to system design and end-user experience. Geek. Linux user.