"Traffic just grew 5×. What do you change?" is a near-guaranteed prompt. This unit is the systematic answer: find the bottleneck, scale the right layer, balance the load.
⏱ 30 minPrereq: Units 02, 04, 06You'll be able to: answer "handle 5× traffic" methodically
By the end of this unit
Choose vertical vs horizontal scaling and know why stateless beats stateful.
Walk a request path and find which layer saturates first.
Optimize read paths vs write paths with the right tool each.
Place load balancers and pick a balancing algorithm.
Vertical vs horizontal
Vertical (scale up)
Bigger machine: more CPU/RAM.
Zero code change; instant relief.
Hard ceiling + single point of failure + expensive.
Requires statelessness, coordination, data partitioning.
The senior framing "I scale out, not up, for anything that must stay available — so I keep app servers stateless (session/state in Redis or a token), put them behind a load balancer, and autoscale on load. Vertical scaling is a quick stopgap with a ceiling."
The 5× method: find the bottleneck first
Don't scale everything — walk the request path and ask what saturates first. Then apply the matching fix:
Layer
Saturation sign
Fix
App tier
CPU-bound, stateless
Add servers + load balancer; autoscale.
Read path
DB read CPU high, read-heavy ratio
Cache (Unit 06) + read replicas (Unit 04).
Write path
Primary write-saturated; data too big
Shard (Unit 04); batch; queue writes (Unit 08).
Slow work
Long requests block threads
Move to async workers via a queue.
Static/media
Bandwidth-heavy assets
CDN + object storage (Units 10–11).
State is the enemy of horizontal scale If app servers hold session state in memory, you can't freely add/remove them. Externalize state (Redis, DB, signed tokens) so any server can handle any request — the precondition for autoscaling.
Read vs write optimization
Read-heavy (most apps)
Cache aggressively (multi-layer).
Read replicas; route reads off the primary.
Denormalize / precompute hot views (e.g., precomputed feeds).
CDN for static and cacheable responses.
Write-heavy (logging, IoT, chat)
Shard to spread write load.
Batch & buffer through a queue.
Write-optimized stores (LSM-tree: Cassandra).
Async/eventual where correctness allows.
Load balancing
A load balancer spreads requests across servers and removes unhealthy ones via health checks. L4 (transport) balances by IP/port — fast and protocol-agnostic. L7 (application) reads HTTP and can route by path/host/header — smarter, slightly costlier. Common algorithms:
Round-robin — simple even spread.
Least-connections — favors the least-busy server; good for uneven request costs.
Consistent hashing / sticky — pins a client/key to a server (useful for cache locality; avoid for state if you can).
The LB itself must not be a single point of failure — run it redundantly (active-active or active-passive) behind a floating IP / DNS.
Quick check
Traffic grows 5×. Your DB primary is at 90% CPU but it's almost all reads, and the dataset fits on one node. First move?
Cache + replicas. It's a read bottleneck on data that still fits one node, so offload reads first — cheaper and simpler than sharding. Reserve sharding for write saturation or size limits. Match the fix to the saturating layer.
Quick check
Why must app servers be stateless to autoscale cleanly?
Any server handles any request. If session state lives in one server's memory, scaling in/out or a server dying breaks users mid-session. Externalizing state (Redis/DB/signed token) lets the LB send a request anywhere — the foundation of elastic scaling.
Practice before you move on
Take your photo-feed design at 1× and narrate exactly what changes at 10×: which layer saturates first, the order you'd apply fixes (cache → replicas → shard → CDN → async), and where load balancers sit. Keep it to a tight 2-minute story.