Caching is the single highest-leverage move in most read-heavy designs. It's also where interviewers test whether you understand the hard parts: invalidation, eviction, and stampedes.
⏱ 30 minPrereq: Units 02–05You'll be able to: place a cache and defend the invalidation strategy
By the end of this unit
Know where caches live (client → CDN → app → DB) and what each layer saves.
Compare memcached vs Redis and pick deliberately.
Explain cache-aside, write-through, and write-back patterns.
Handle the three hard problems: invalidation, eviction at max size, and stampedes.
Why cache: the latency argument
From Unit 02: memory is ~1000× faster than SSD and ~100× faster than spinning disk. A cache keeps hot data in RAM so most reads never touch the database. With a 95% hit rate, you cut database read load by 20× — often the difference between one database and a sharded fleet.
Each layer catches reads earlier. A request ideally never reaches the DB. Caches compound.
memcached vs Redis
memcached
Pure in-memory key→value (strings/blobs).
Multithreaded, dead simple, very fast.
No persistence, no rich types.
Pick when you want a plain, huge, volatile cache.
vs
Redis
Data structures: lists, sets, sorted sets, hashes, streams.
Great for leaderboards, rate limiters, queues, feeds.
Default modern choice unless you need nothing but raw KV.
Soundbite "I'd use Redis — I want sorted sets for the feed/leaderboard and TTLs for expiry. If I only needed a dumb, massive object cache I'd consider memcached for its simplicity and multithreaded throughput."
Read & write patterns
Pattern
How it works
Tradeoff
Cache-aside (lazy)
App checks cache; on miss, reads DB and populates cache.
Most common. Only caches what's used, but first read is a miss and stale risk on writes.
Write-through
Writes go to cache and DB synchronously.
Cache always fresh; writes are slower.
Write-back (write-behind)
Write to cache; flush to DB asynchronously.
Fast writes, risk of data loss if cache dies before flush.
Cache-aside + a TTL is the default you should reach for. Mention write-through when freshness matters and write-back when write latency dominates and you can tolerate some loss.
The three hard problems
1 · Invalidation — "the second-hardest problem in CS"
When the underlying data changes, stale cache entries must go. Options: TTL (expire after N seconds — simple, accepts bounded staleness), explicit invalidation (delete/update the key on write — fresh but easy to miss a path), or versioned keys (key includes a version so old entries are simply never read). Most designs combine a short TTL with explicit invalidation on the obvious write paths.
2 · Eviction — reaching max storage
A cache has finite RAM. When full, it evicts by policy: LRU (least-recently-used — the common default), LFU (least-frequently-used), or TTL/random. The metric that matters is hit rate; size the cache so the working set fits and watch for thrash when it doesn't.
3 · Stampede / thundering herd
A hot key expires and thousands of requests miss simultaneously, all hammering the DB at once. Mitigations: request coalescing (one fetch fills the cache while others wait), staggered/jittered TTLs, and early/background refresh before expiry. Mentioning stampedes unprompted is a strong senior signal.
Don't forget consistency A cache is a second copy of your data, so it reintroduces the consistency questions from Unit 05. Always say what staleness window your cache tolerates and why it's acceptable for that feature.
Quick check
A single trending post's cache entry expires and 50,000 requests/sec immediately hit the database. This is a…
Stampede. The issue is many simultaneous misses on one hot key, not capacity. Coalesce so a single request repopulates the cache while the rest wait, and add jitter so popular keys don't all expire at once.
Quick check
Which pattern keeps the cache guaranteed-fresh at the cost of slower writes?
Write-through. Every write updates cache and DB together, so reads never see stale data — but writes pay the cost of two synchronous updates. Write-back is the opposite tradeoff (fast writes, risk of loss); cache-aside leaves freshness to TTL/invalidation.
Practice before you move on
Add caching to the photo-feed design. Decide: what to cache (rendered feed? post objects? counts?), which pattern, TTL vs explicit invalidation when a user posts, eviction policy, and how you'd protect a celebrity's post from a stampede. Write it as you'd say it in 90 seconds.