One database box has a ceiling. This unit is how you blow past it: split data across machines (sharding), copy it for reads and safety (replication), and route through proxies.
⏱ 30 minPrereq: Unit 03You'll be able to: scale a database past one node and survive failures
By the end of this unit
Distinguish replication (copies) from sharding (splits) — and why you usually need both.
Choose a sharding key and explain hash vs range vs consistent hashing.
Reason about primary/replica, replication lag, and failover.
Place a proxy/load balancer in front of your data tier and say why.
Two different problems
Replication = copies
Same data on multiple nodes.
Solves: read scale, availability, durability.
Primary takes writes; replicas serve reads.
Does not help if one node can't hold all the data.
+
Sharding = splits
Each node holds a subset of the data.
Solves: write scale, total storage size.
Routing layer sends each key to the right shard.
Does not by itself give you redundancy.
Real systems combine them: shard for size/write-scale, then replicate each shard for reads and failover.
Choosing a shard key
The shard (partition) key decides which node owns a row. Pick it to (a) spread load evenly and (b) keep your common query on a single shard.
Strategy
How
Watch out for
Hash(key)
shard = hash(id) % N
Even spread, but % N means adding a node reshuffles almost everything.
Range
A–M on shard 1, N–Z on shard 2
Simple range scans; risks hot shards (e.g., everyone named with recent timestamps).
Consistent hashing
keys + nodes on a ring
Adding/removing a node moves only ~1/N of keys. The standard answer for elastic clusters.
Directory / lookup
a service maps key→shard
Most flexible; the lookup table is now a dependency to keep available.
The hot-shard trap Sharding by celebrity_id or by raw timestamp concentrates load on one node. Mention skew and how you'd mitigate it (compound keys, salting, separate handling for whales) — interviewers probe for this.
What sharding costs you Cross-shard joins and transactions become hard or impossible; you do them in app code or avoid them by modeling. Re-sharding is operationally painful. Only shard when a single node genuinely can't cope — don't shard prematurely.
Replication: sync vs async, and failover
Synchronous
Primary waits for replica to confirm before acking the write.
No data loss on failover; higher write latency.
vs
Asynchronous
Primary acks immediately; replicas catch up.
Fast writes, but a crash can lose the last few writes (replication lag).
Redundancy is the goal replication serves: no single node's death loses data or takes you down. On primary failure, a failover promotes a replica to primary (automatically via a coordinator, or manually). Multi-leader and leaderless (Dynamo-style quorum) setups trade more write availability for more conflict-resolution complexity.
Read-your-writes gotcha If a user writes to the primary then reads from a lagging replica, they may not see their own change. Fixes: route that user's reads to the primary briefly, or read from a replica known to be caught up. Naming this wins points.
Proxies in front of the data tier
A database proxy (ProxySQL, Vitess, PgBouncer, or a routing layer you describe) sits between app servers and the database to: route reads to replicas and writes to the primary, pool connections, hide shard topology from the app, and enable failover without app changes. In a diagram, draw it as the box the app talks to so the app never hardcodes a shard address.
Quick check
Your single database is at 95% CPU on writes, and the dataset no longer fits on one disk. What do you reach for first?
Shard. Read replicas only help reads and don't increase total storage or write capacity — every replica still takes every write and holds the whole dataset. Write saturation + size limits is the textbook signal to partition the data.
Quick check
Why is consistent hashing preferred over hash(key) % N for an elastic cluster?
Minimal remapping. With % N, changing N reshuffles almost every key — catastrophic for a live system. Consistent hashing moves only the keys near the changed node. Virtual nodes are added on top to smooth out distribution.
Practice before you move on
Design the data tier for a chat app storing billions of messages. Pick a shard key (hint: by conversation), explain how you'd serve "last 50 messages in a conversation" from one shard, and describe your replication + failover plan. Call out one hot-shard risk and your mitigation.