Zero→Hero
0/17
Module 4 · Operate & Architect · Unit 15

Feed Generation (Deep Dive)

The news-feed/timeline is the most-asked deep dive in the genre. It ties together caching, queues, sharding, and consistency into one hard, beautiful tradeoff: pull vs push.

⏱ 30 min Prereq: Units 04, 06, 08, 09 You'll be able to: design a timeline and defend fan-out choices

By the end of this unit

The core question

When user A posts, everyone who follows A should see it in their feed. The whole design hinges on when you do the work of assembling each follower's feed: at write time (push the post into followers' feeds now) or at read time (gather posts when a follower opens the app). Feeds are extremely read-heavy (Unit 02), so this is a deliberate amplification tradeoff.

Push vs pull

Fan-out on write (push)

  • On post, write the post id into every follower's precomputed feed list (in Redis).
  • Reads are O(1): just read your ready-made list. Blazing-fast reads.
  • Writes amplify hugely: 1 post by someone with 10M followers = 10M list writes.
  • Wastes work for inactive followers.
vs

Fan-out on read (pull)

  • On post, just store the post once.
  • On read, fetch recent posts from everyone you follow and merge. Cheap writes.
  • Reads are expensive: query & merge across many followees every time.
  • Scales writes well; hurts read latency for users following many people.
PUSH: work at write time A F1 feed F2 feed F3 feed PULL: work at read time F A posts B posts C posts
Push pays at write to make reads trivial; pull keeps writes cheap but does the merge work on every read.

The hybrid (what real systems do)

Neither pure approach survives a celebrity with tens of millions of followers — push would generate tens of millions of writes per post (the hot-key / fan-out storm). The standard answer:

Say the hybrid out loud "I'd fan out on write for typical users so reads are O(1) from Redis, but for high-follower accounts I'd skip fan-out and merge their recent posts in at read time. That caps write amplification while keeping reads fast for everyone." This is the answer interviewers are listening for.

Ranking, aggregation & pagination

Quick check
Why does pure fan-out-on-write break for celebrity accounts?
Write amplification. Pushing a single post into 10M follower feeds is 10M writes per post — a storm that can overwhelm the system. The hybrid fixes it by pulling celebrity posts at read time instead of fanning them out.
Quick check
A user who follows very few, low-activity accounts opens the app. Which approach serves them most efficiently?
Pull is fine here. When a user follows few, rarely-posting accounts, fanning out on write wastes effort; merging a handful of sources at read time is cheap. This is the flip side of why the hybrid keys on follower/activity counts — match the strategy to the case.

Practice before you move on

Design "home timeline" end to end: storage for posts and for precomputed feeds, the push path for normal users, the pull merge for celebrities, the threshold you'd pick, how you paginate, and where caching helps. Time yourself to 4 minutes — this is exactly the deep dive you'll get.

Finished this 30-min unit?