"Design the Facebook news feed" or "design the Twitter timeline" is the classic fan-out question. A user posts once; hundreds or millions of people should see it in their feed. You can do the work when the post is written, or when each feed is read, and each choice is perfect for one kind of user and terrible for another. The answer interviewers want is the hybrid, and the follow-ups test whether you can say exactly where the line goes and why.
Users follow other users and see their posts in a home feed, newest or most relevant first, with infinite scroll. Posts have text and optional images.
In scope
- Publishing a post, building and reading the home feed, pagination
- Accounts with very different follower counts
- Ranking, at a high level
A news feed shows each user posts from the people they follow in under 200 ms. At an assumed 100M daily users that is about 11,600 feed loads a second, and pushing 10M daily posts to 200 followers each is 2 billion inserts a day, about 800 GB of feed cache, while one 50M-follower post would take over 8 minutes to push. The post service publishes to Kafka, fan-out workers push post IDs into per-user Redis lists, and the feed service merges in celebrities' posts pulled at read time, hydrates from a separate post cache, filters and ranks. The deep dives place the push-pull threshold, skip inactive users, use cursors instead of offsets and keep ranking to tens of milliseconds; failures cover fan-out lag, a lost cache shard and hot celebrity posts.
Phase 1: Requirements (3 minutes)#
"Is the feed chronological or ranked? How quickly must a new post appear in followers' feeds? Is there a follower cap?"
"Start chronological, then tell me where ranking fits. A few seconds' delay is fine. No cap: some accounts have 50 million followers."
Functional requirements#
- Publish a post (text, image references)
- Follow and unfollow
- Get the home feed: posts from followed accounts, newest first (ranking later), paginated
- Deleted posts and blocked users never appear
Non-functional requirements#
- Fast reads: feed load under 200 ms at p99 (a design target)
- Read heavy: far more feed loads than posts
- Eventually consistent: a new post may take a few seconds to reach followers' feeds
- Highly available: a slightly stale feed beats an error page
Phase 2: Estimation (3 minutes)#
- Feed loads: assume each of 100M DAU opens the feed 10 times a day: 1B a day ≈ ~11,600/s, ~35,000/s at a 3x peak
- Posts: assume 10% of DAU post once a day: 10M posts a day ≈ ~116/s
- Fan-out on write: assume 200 followers per post on average: 10M × 200 = 2B feed inserts a day ≈ ~23,000/s
- Feed cache: keep the newest 500 post IDs per user at ~16 bytes each (ID + timestamp) = 8 KB per user; for 100M users ≈ 800 GB, a sharded Redis cluster
- The outlier: one post by a 50M-follower account is 50M inserts. At an assumed 100,000 inserts a second for the fan-out workers, that one post takes over 8 minutes to deliver
"The averages say fan-out on write is affordable. The outlier says it can't be the only strategy."
Phase 3: API (2 minutes)#
POST /v1/posts
{ "text": "Shipped it!", "media_ids": ["m_91"] }
→ 201 { "post_id": "p_240044212224069649" }
GET /v1/feed?limit=20
→ 200 { "items": [ { "post_id": "p_...", "author": {...}, "text": "...", "created_at": "..." }, ... ],
"next_cursor": "eyJ0cyI6MTc5MzE4MTY0MSwiaWQiOiJwXzI0MDA0In0" }
GET /v1/feed?limit=20&cursor=eyJ0cyI6... # the next page
POST /v1/users/u_42/follow DELETE /v1/users/u_42/followThe cursor encodes the position (timestamp and post ID of the last item seen), not an offset. Deep dive 2 explains why.
Phase 4: High-level design (4 minutes)#
%% title: News feed: write path and read path
flowchart TD
U@{ icon: "sd:users", label: "Users" } --> LB@{ icon: "sd:load-balancer", label: "Load balancer" }
LB -->|"POST /posts"| PS["Post service"]:::core
LB -->|"GET /feed"| FS["Feed service<br/>merge, hydrate, rank"]:::hot
PS --> PDB@{ icon: "sd:database", label: "Posts DB<br/>sharded by post ID" }
PS -->|"post created"| K@{ icon: "tech:kafka", label: "Kafka · new posts" }
K --> FO["Fan-out workers<br/>skip celebrities and inactive users"]:::service
FO --> G@{ icon: "sd:hexagons", label: "Social graph<br/>followers, following" }
FO -->|"push post ID"| FC@{ icon: "tech:redis", label: "Feed cache (Redis)<br/>500 IDs per user" }
FS --> FC
FS -->|"pull: celebrities' recent posts"| PC@{ icon: "tech:redis", label: "Post cache (Redis)<br/>post objects, author timelines" }
PC --> PDB
FS --> G
class U client
class LB edge
class PDB store
class K queue
class G service
class FC,PC cache- Post service stores the post and publishes a "post created" event. The author gets a 201 immediately; fan-out happens asynchronously.
- Fan-out workers read the author's followers from the social graph and prepend the post ID to each follower's feed list in Redis (a sorted set trimmed to 500 entries), except when the author is a celebrity or the follower is inactive.
- Feed service reads the user's precomputed list, merges in recent posts from followed celebrities, hydrates IDs into full posts with a multi-get from the post cache, filters deleted and blocked content, and returns a page.
- Two caches, two jobs: the feed cache holds only IDs per user; the post cache holds each post object once. An edit or delete updates one post object, not millions of feed entries.
Phase 5: Deep dives (5 minutes)#
Deep dive 1: Push, pull, or both#
Push (fan-out on write)
- On post: write the ID into every follower's feed
- Reads are one cache lookup: fast
- A 50M-follower post is 50M writes and minutes of lag; work wasted on followers who never log in
Pull (fan-out on read)
- On read: fetch recent posts from everyone you follow and merge
- No write amplification, always fresh
- A user following 2,000 accounts triggers 2,000 lookups per feed load, at 35,000 loads a second
Hybrid
- Push for ordinary authors, pull for celebrities
- Most feeds are one cache read plus a merge of a few celebrity timelines
- One tunable: the follower count where an author stops being pushed
- Pick the threshold: for example, authors with more than 10,000 followers are not pushed. Tune it from the fan-out lag and read cost you observe.
- Celebrity posts go to the author's own timeline in the post cache (their newest posts, kept hot because many people read them).
- On read, merge: the user's pushed list plus the newest posts from each celebrity they follow (usually a few), sorted by time.
- Skip inactive followers: don't push to users who haven't opened the app in, say, 30 days. When they return, build their feed by pull once, then resume pushing.
Deep dive 2: The read path in 200 ms#
"Why a cursor? ?page=3 is simpler."
"Feeds move. If five new posts arrive while you read page 1, offset-based page 2 starts five items earlier and shows you five posts again. A cursor says 'items older than this timestamp and ID', which stays correct however many new posts arrive above it."
- Read the ID list from the feed cache:
ZREVRANGEBYSCORE feed:u_42 (<cursor_ts> -inf LIMIT 0 20(one call). - Merge celebrity posts newer than the cursor from their timelines (a few calls, in parallel).
- Hydrate the 20 IDs with one multi-get to the post cache, and one to the user cache for author names and avatars.
- Filter posts that were deleted, from blocked users or from accounts the user has since unfollowed.
- Rank (if not chronological) and return 20 items plus the next cursor.
A ranked feed takes a larger candidate set (say the newest 500 IDs plus celebrity posts) and scores each one, for example by recency, how often the user interacts with the author, and the post's early engagement. Scoring a few hundred candidates per load is cheap; the expensive part is keeping the features (affinity, engagement counts) precomputed and cached, so ranking adds tens of milliseconds, not hundreds.
Fan-out and pagination
Q1/2 Why not push every post to every follower's feed?
Answer & explanation
Answer: C
Push makes reads cheap but write cost grows with follower count. Above a threshold, pull those authors' posts at read time instead.
Q2/2 New posts arrive while a user scrolls. Why does a cursor beat
?page=2?Answer & explanation
Answer: B
"Older than this timestamp and ID" is stable no matter how many posts land above it. An offset counts from the top, which keeps moving.
Phase 6: Trade-offs and failure modes (2 minutes)#
Fan-out falls behind
A burst of posts (a big live event) grows the Kafka backlog, and feeds lag. Monitor consumer lag in seconds, scale the workers, and lower the celebrity threshold temporarily so fewer authors are pushed.
The feed cache loses a shard
Feeds for those users are rebuilt by pull on their next load. If a whole shard's users come back at once, that is a thundering herd on the posts DB: rebuild with request coalescing and a rate limit, and serve a short "recent posts" fallback meanwhile.
A celebrity post is a hot key
Millions of feed loads read the same post object in the same minute. Replicate hot post objects across cache nodes, or keep them in a small in-process cache in the feed service.
Unfollow, delete, block
Pushed IDs already sit in feeds. Filter at read time (cheap: the post cache knows deletions, the graph knows follows and blocks), and clean feed lists lazily.
Consistency
The author should see their own post immediately: insert it into the author's own feed synchronously, or merge their latest posts at read time (read-your-writes). Everyone else can wait a few seconds.
Phase 7: Go deeper in the course#
- Push, pull, hybrid and production feed caching in depth: Week 4 · Day 4: Feed Caching
- A full timeline system at Twitter scale: Side quest: Twitter / X
- Celebrity accounts as hot keys: Week 1 · Day 4: Hot Keys and Skew
- Rebuilding caches without a stampede: Week 4 · Day 3: Thundering Herd
- Edits, deletes and stale entries: Week 4 · Day 2: Invalidation Strategies
- The post-created event stream: Week 3 · Day 1: Queue vs Stream
- Read-your-writes for the author: Week 5 · Day 1: Consistency Models
Final check#
Final check · News feed
Q1/3 10M posts a day with 200 followers each, all pushed. About how many feed inserts a second is that?
Answer & explanation
Answer: B
10M × 200 = 2B inserts a day; 2,000,000,000 ÷ 86,400 ≈ 23,000 a second on average.
Q2/3 Why keep post IDs in the feed cache and post objects in a separate cache?
Answer & explanation
Answer: C
Each post is stored once and referenced by ID from many feeds, so changes touch one place and feeds stay small (about 8 KB per user).
Q3/3 A user who hasn't opened the app in 60 days returns. What does the hybrid design do?
Answer & explanation
Answer: B
Inactive users are skipped by fan-out to save work. On return, a one-time pull rebuilds the feed and the user rejoins the push path.
New lessons by email
New lessons, side quests and case files as they ship, at most once a week. No spam, unsubscribe anytime.
Discussion · …
Public: everyone reading this lesson sees comments. Be kind and keep it on the material.