In previous article
, I explained you how to create a service like TinyURL.
That kind of service requires storing a lot of data what implies sharding this data across multiple DB instances.
I showed a really simple algorithm of how data could be distributed in that case and briefly mention something about replication.
Now the question is what would we do in real life?
Real-life distribtued DB
Using existing NoSQL solution would be a good choice as they already solve main problems with distributed DB systems:
- Data replication and durability
- High availability even in corner cases like adding new DB shard to the system and hardware failures
- Scalability and capacity management
Examples of such systems are AWS DynamoDB and Google Cloud Bigtable.
Both are very similar but store data in a slightly different way hence give different guarantees.
I would like to focus on the first one today.
Now, I would like to talk a little bit more about DynamoDB system design as this is the most interesting part for engineers.
How is it build?
How data is distributed and replicated?
What happens in case of hardware failure?
Unfortunately, “jasso” - Senior Principal Engineer in AWS already did that on his presentation “Amazon DynamoDB Under the Hood: How We Built a Hyper-Scale Database” during re:Invent 2018.
The presentation have a ton of knowledge on how to design a mature large-scale system. Enjoy!
Article by Tomasz Kuczma
Software engineer with a passion. Interested in computer networks and large-scale distributed computing.
He loves to optimize and simplify software on various levels of abstraction starting from memory ordering through non-blocking algorithms up to system design and end-user experience. Geek. Linux user.
The views I express are my alone and they do not necessarily express the views of my employer or ex-employers.
They are not investment advice nor based on any non-public information of any kind.
Poglądy, które wyrażam, są tylko moje i niekoniecznie wyrażają opinie mojego pracodawcy lub byłych pracodawców.
Nie są poradami inwestycyjnymi ani nie opierają się na jakichkolwiek niepublicznych informacjach.