~learn.

Instagram Feed and Stories, the whole design.

A ranked feed and a stories tray for 3 billion monthly users, by storing every photo once, moving only small time-sortable IDs, and spending the compute on choosing which twenty posts to show.

6 diagrams · 8 sections · revise in ~10 min · Download PDF prints it in the light theme, diagrams included

SQ-21 · Instagram Feed and Stories · design sheethimanshukukreja.in/learn/sheets/bonus-21-instagram-feed

At a glance#

Number Value What it drives
Users 3B monthly · 1.5B daily (assumed half) Meta doesn't publish daily or posting figures
Feed pages ~700,000 / s at peak 10 opens × 2 pages a day each (assumed)
Reads per post ~300 30B pages vs 100M posts a day: write once, read cheaply
Ranking ~350M model scores / s 500 candidates a page; a cheap first stage first
Fan-out ~20B inbox writes / day ~200 followers a post; huge accounts pulled at read
Stories 500M daily users (2019) · ~15B seen-marks / day seen state is the write load
Media ~5.7 PB new / day · ~8 Tbps at peak store once, resize, CDN, hot then warm tiers
Feed before ranking ~70 % missed (2016) why the feed is ranked at all

Architecture#

DiagramFeed and stories on one page: bytes, IDs, decisions
%% title: Feed and stories on one page: bytes, IDs, decisions
flowchart TD
  APP@{ icon: "sd:phone", label: "Android · iOS · web apps<br/>prefetch next images and first frames" } ==>|"media"| CDN@{ icon: "sd:cdn", label: "CDN<br/>right-sized images · video segments" }
  APP ==> API@{ icon: "sd:api", label: "API gateway<br/>post · feed · tray" }
  API -->|"upload"| MED["Media service<br/>original kept · 4 sizes · encodings"]:::service
  MED --> BLOB@{ icon: "sd:object-storage", label: "Blob storage<br/>content-addressed · hot → warm" }
  API -->|"create post"| POST@{ icon: "tech:postgresql", label: "PostgreSQL · posts<br/>64-bit IDs · sharded by author" }
  POST -.->|"new post"| FAN@{ icon: "sd:worker", label: "Fan-out workers<br/>push IDs · skip huge accounts" }
  FAN ==> INB@{ icon: "tech:cassandra", label: "Cassandra · inboxes<br/>newest 1,500 IDs per viewer" }
  API ==>|"open feed"| FEED["Feed service<br/>filters → light model → heavy model → rules"]:::hot
  FEED --> INB
  FEED --> FS@{ icon: "tech:redis", label: "Redis · features<br/>counts · closeness · seen" }
  APP -->|"open tray"| TRAY["Stories tray<br/>24 h index · seen set · audience"]:::core
  CDN -.-> BLOB
  class APP client
  class CDN,API edge
  class BLOB,POST,INB store
  class FAN service
  class FS cache
DiagramThe feed ranking funnel for one page
%% title: The feed ranking funnel for one page
flowchart TD
  C(["1,500 candidates<br/>inbox ∪ huge accounts followed"]):::client --> F1["Filters<br/>seen · blocked · muted · deleted"]:::service
  F1 --> S1["First stage · light model<br/>~0.01 ms each · 1,200 → 300"]:::core
  S1 --> S2["Second stage · multi-task model<br/>time · comment · like · share · profile tap"]:::hot
  S2 --> V["Value = weighted sum<br/>of the five predictions"]:::core
  V --> RR["Final rerank<br/>≤ 2 in a row per author · integrity · freshness"]:::core
  RR --> OUT(["Page of 10<br/>logged as impressions"]):::ok
  S2 -.->|"slow or down"| FB(["Serve first-stage order"]):::bad

Key flows#

DiagramA baby photo, from posting to the top of Meera's feed
%% title: A baby photo, from posting to the top of Meera's feed
sequenceDiagram
  autonumber
  participant C as Cousin's app
  participant M as Media service
  participant P as Posts shard
  participant F as Fan-out
  participant R as Feed service
  participant Me as Meera's app
  C->>M: Upload photo · 3.2 MB
  M-->>C: Keys for original + 1080 · 640 · 320 · thumb
  C->>P: Create post with media keys
  P-->>C: Time-sortable post ID
  alt author below 1M followers
    P-)F: Push ID to 420 inboxes
  else huge account
    P-)F: Recent-posts cache only · readers pull
  end
  Me->>R: Open feed · three hours later
  R->>R: Inbox ∪ huge accounts · filter · rank in stages
  R-->>Me: Page 1 · the baby photo first
  Me->>Me: Fetch the 1080 copy from the CDN
DiagramBuilding one stories tray
%% title: Building one stories tray
sequenceDiagram
  autonumber
  participant Me as Meera's app
  participant T as Tray service
  participant A as Active-story index
  participant S as Seen state
  Me->>T: Open app
  T->>A: Which of 600 followed accounts have stories · 90
  T->>T: Audience check · close friends only if listed
  T->>S: Which has Meera seen · 30 fully
  T-->>Me: 60 unseen first, seen ones last
  Me->>Me: Prefetch first frames of the top 3
  Me->>S: Seen-marks, batched every few seconds
  Note over A: at hour 24 the entry expires · story leaves every tray · media stays in the archive

Data model#

DiagramCore entities
%% title: Core entities
classDiagram
  class Post {
    +postId : int64 · 41 time · 13 shard · 10 seq
    +author : user
    +mediaKeys : list of keys
    +caption : string
    +deleted : bool · tombstone
  }
  class Media {
    +key : hash of the bytes
    +sizes : 1080 · 640 · 320 · thumb
    +tier : hot / warm
  }
  class Inbox {
    +viewer : user
    +postIds : newest 1,500
  }
  class Story {
    +storyId : int64
    +author : user
    +audience : everyone / close friends
    +expiresAt : posted + 24 h
  }
  class SeenSet {
    +viewer : user
    +storyIds : set · 48 h expiry
  }
  class Follow {
    +follower : user
    +followee : user
  }
  Post "1" --> "many" Media : shows
  Inbox "many" --> "many" Post : candidate IDs
  Story "1" --> "1" Media : shows
  SeenSet "many" --> "many" Story : marks
  Follow "many" --> "1" Inbox : fills

Lifecycle#

DiagramA story from posting to the archive
%% title: A story from posting to the archive
stateDiagram-v2
  [*] --> UPLOADING
  UPLOADING --> FAILED: upload not acknowledged
  UPLOADING --> ACTIVE: media stored, index entry added
  ACTIVE --> ACTIVE: viewed, seen-marks written
  ACTIVE --> HIDDEN: author deletes it
  ACTIVE --> EXPIRED: 24 hours pass, index entry lapses
  EXPIRED --> ARCHIVED: media to warm storage
  ARCHIVED --> [*]
  HIDDEN --> [*]
  FAILED --> [*]
  class ARCHIVED ok
  class FAILED,HIDDEN bad
  class ACTIVE queue

Decisions#

Decision Chose Over Because
Media storage Original + derived sizes, content-addressed Resize every request stored once; caches never stale
Video encodings Repackage basic ABR, advanced by watch time Encode every version basic cost −94 %, advanced coverage +33 %
IDs 41 time · 13 shard · 10 seq, made in the database A central ID service time-sortable, shard inside, no new moving parts
Fan-out Push for most, pull for huge accounts Push for everyone 400M inbox writes take ~7 minutes
Feed order Staged ranking on five predictions Chronological people missed ~70 % of their feeds
Story expiry Index entry with 24 h TTL Delete jobs at hour 24 leaves every tray at once; media archived
Deletes and privacy Tombstone + checks at hydrate Purge every inbox first instant, nothing leaks from stale inboxes

When it breaks#

Failure What users see Contain it with
A 400M-follower account posts nothing unusual no fan-out; merge at read; pre-warm CDN edges
Heavy ranking model slow or down a slightly worse feed serve first-stage order; newest-first as last resort
Fan-out queue backs up new posts arrive late reads still work from existing inboxes
CDN hit rate falls to 80 % slow images ~6× origin load; pre-warm, add edge capacity
Seen-mark writes spike 10× a story looks unseen again batch longer on the phone; loss is harmless
Post deleted after fan-out it vanishes at once tombstone at hydrate; lazy inbox and media cleanup

Say it in the interview#