"Design WhatsApp" or "design Messenger" changes the shape of the usual interview. Most systems answer requests; a chat system holds millions of connections open and pushes messages down them. The questions are about state you can't put behind a load balancer: which server is Bob connected to right now, what happens to Alice's message if Bob is offline, and how the ticks know the difference between "on our server" and "on Bob's phone".
Build the messaging backend for a mobile chat app: one-to-one chats, group chats up to 500 members, delivery and read receipts, online presence, and message history on every device.
In scope
- Real-time delivery, offline delivery, receipts, presence
- Multiple devices per user
- Storage of message history
A chat backend delivers one-to-one and group messages to several devices per user with sent, delivered and read ticks. At an assumed 2B messages a day (about 23,000 a second) and 10M open connections, about 200 stateful WebSocket gateways register each device in a Redis session registry, while a stateless chat service assigns each message the next sequence number in its conversation, stores it in Cassandra before acknowledging, routes it to the recipients' gateways, and falls back to push notifications for offline devices. The deep dives cover client message IDs for safe retries, cumulative receipts, sync by last sequence, group fan-out versus pull for huge channels and cheap presence; failures cover gateway crashes, jittered reconnects during deploys and what end-to-end encryption changes.
Phase 1: Requirements (3 minutes)#
"One-to-one and groups? How big can groups get? Multiple devices per account? Do we keep history on the server, or only until delivery? Is end-to-end encryption in scope?"
"Both, groups up to 500. Phone plus web and desktop. Keep history on the server. Mention encryption in trade-offs, but design the plain version first."
Functional requirements#
- Send and receive one-to-one and group messages (text, plus references to media in object storage)
- Online recipients get the message instantly; offline recipients get a push notification and the message when they reconnect
- Receipts: sent (stored on the server), delivered (reached a recipient device), read
- Presence: online, or last seen
- History synced across a user's devices
Non-functional requirements#
- Low latency: under 500 ms end to end when both users are online (a design target)
- No loss: once the sender sees "sent", the message is durable
- Ordered: messages in a conversation appear in the same order on every device
- No duplicates shown: retries must not show a message twice
- Highly available: a gateway crash disconnects some users briefly; it doesn't lose messages
Phase 2: Estimation (3 minutes)#
- Messages: assume 50M DAU × 40 messages sent a day = 2B a day ≈ ~23,000/s, ~70,000/s at a 3x peak
- Connections: assume 20% of DAU are online at peak = 10M open WebSocket connections
- Gateways: if one server comfortably holds ~50,000 connections, that's ~200 gateway servers, plus headroom for failures and deploys
- Storage: assume ~200 bytes per message with metadata: 2B × 200 B = ~400 GB a day, ~146 TB a year before replication
- Group fan-out: a message to a 500-member group becomes up to 500 deliveries, so deliveries per second can be several times the send rate
"The unusual numbers are the connections. 10M long-lived connections means the gateways are stateful, and routing a message means knowing which gateway holds the recipient."
Phase 3: API (2 minutes)#
Clients keep one WebSocket open; history and catch-up use plain HTTPS.
// client → server
{ "type": "send", "client_msg_id": "c-9f1", "conv_id": "cv_77", "body": "on my way" }
// server → sender: stored, with its place in the conversation
{ "type": "ack", "client_msg_id": "c-9f1", "msg_id": "m_51x", "seq": 1042 }
// server → recipient devices
{ "type": "msg", "conv_id": "cv_77", "seq": 1042, "from": "u_alice", "body": "on my way" }
// recipient device → server (receipts are cumulative: everything up to this seq)
{ "type": "delivered", "conv_id": "cv_77", "up_to_seq": 1042 }
{ "type": "read", "conv_id": "cv_77", "up_to_seq": 1042 }GET /v1/sync?cursor=<opaque> → messages since the device's last sync
GET /v1/conversations/cv_77/messages?before_seq=1000&limit=50 → scroll back through historyPhase 4: High-level design (4 minutes)#
%% title: Chat backend: stateful gateways, stateless chat service
flowchart TD
A@{ icon: "sd:phone", label: "Alice's phone" } <-->|"WebSocket"| G1["Gateway 1<br/>~50,000 connections"]:::core
B@{ icon: "sd:laptop", label: "Bob's laptop" } <-->|"WebSocket"| G2["Gateway 2"]:::core
G1 --> CS["Chat service<br/>assign seq, store, route"]:::hot
CS --> M@{ icon: "tech:cassandra", label: "Cassandra · messages<br/>partition: conversation" }
CS --> SR@{ icon: "tech:redis", label: "Redis · sessions<br/>user → gateways, devices" }
CS -->|"deliver to Bob's devices"| G2
CS -->|"recipient offline"| PN@{ icon: "sd:bell", label: "Push service" }
PN --> APNS@{ icon: "tech:apple", label: "APNs" }
PN --> FCM@{ icon: "tech:firebase", label: "FCM" }
PR["Presence service<br/>heartbeats"]:::service --> SR
G1 -.->|"heartbeats"| PR
class A,B client
class M store
class SR cache
class PN service
class APNS,FCM external- Gateways terminate WebSockets and are the only stateful part. They register each connection in the session registry and forward frames to the chat service. A layer-4 load balancer spreads new connections across them.
- Chat service is stateless: it assigns the message its sequence number in the conversation, writes it, acknowledges the sender, looks up recipients' devices in the session registry, and forwards the message to the gateways that hold them.
- Message store: a wide-column store suits the access pattern, "messages of one conversation, by sequence". Partition by conversation (with time buckets for very long conversations), cluster by
seq. - Session registry:
user → [(device, gateway)]in Redis, with a TTL refreshed by heartbeats so a crashed gateway's entries expire. - Push service: if a recipient has no live connection, send a push notification through APNs or FCM; the message itself is fetched when the app reconnects.
%% title: Alice sends, Bob is online sequenceDiagram autonumber participant A as Alice participant G1 as Gateway 1 participant C as Chat service participant S as Message store participant G2 as Gateway 2 participant B as Bob A->>G1: send (client_msg_id c-9f1) G1->>C: forward C->>S: write seq 1042 C-->>A: ack seq 1042 (one tick: sent) C->>G2: deliver seq 1042 G2->>B: msg seq 1042 B-->>C: delivered up to 1042 C-->>A: receipt (two ticks: delivered)
Phase 5: Deep dives (5 minutes)#
Deep dive 1: Ordering, duplicates and the offline phone#
"Alice's network drops right after she taps send. Her app retries. Bob's phone was offline for an hour. Walk me through what each device sees."
- The client names the message: Alice's app generates
client_msg_idonce and reuses it on every retry. The chat service remembers recent IDs per sender, so a retry returns the sameseqinstead of creating a second message. - The server orders it: the chat service assigns the next
seqfor the conversation (an atomic counter per conversation). Every device sorts byseq, so everyone sees the same order regardless of arrival order. - Store, then acknowledge: the "sent" tick means the message is durable in the store. Nothing the sender sees as sent can be lost.
- Deliver or notify: online devices get it through their gateway; offline ones get a push notification.
- Catch up by sequence: each device remembers the last
seqit has per conversation (or a per-user sync cursor). On reconnect it asks for everything after that, so an hour offline is one sync call, not a replay of pushes. - Receipts are cumulative: "delivered up to 1042" covers 1040 and 1041 too, which keeps receipt traffic low and tolerates lost receipt frames.
"Delivery to devices is at-least-once; a frame can be resent after a reconnect. Because each message carries its seq, the device drops anything it already has. At-least-once plus dedupe looks exactly-once to the user."
Deep dive 2: Finding Bob, groups, and presence#
- Register on connect: when Bob's laptop connects to gateway 2, the gateway writes
u_bob → (laptop, gw-2)to the session registry with a TTL. - Look up on send: the chat service reads Bob's entries and gets every device and gateway he is on (phone on gateway 7, laptop on gateway 2).
- Forward: it sends the message to each of those gateways over an internal RPC (or a per-gateway channel on a pub/sub bus), and each gateway pushes it down the right socket.
- Multi-device: Alice's own other devices are recipients too, so her laptop shows the message she sent from her phone.
Groups up to 500: fan out on write
- Store the message once in the group's conversation partition
- Look up the members' sessions and deliver to each online device
- Offline members catch up by
seqlike any conversation
Very large channels: pull
- Fan-out to 100,000 members per message is a burst nobody needs live
- Deliver only to members who have the channel open; others see a badge and pull on open
- The same idea as the celebrity case in a news feed
Presence changes far more often than messages arrive. Have clients send a heartbeat every 30 s (an assumption), mark a user offline after about two missed heartbeats, debounce flapping connections, and send presence updates only to contacts who currently have that chat open. Everyone else reads "last seen" on demand.
Delivery and routing
Q1/2 Alice's app retries a send after a network drop. What stops Bob from seeing the message twice?
Answer & explanation
Answer: B
The client names the message once. The server maps a repeated
client_msg_idto the existing message andseq, and devices drop sequences they already have.Q2/2 How does the chat service know where to deliver a message for Bob?
Answer & explanation
Answer: C
Gateways register
user → (device, gateway)with a TTL on connect. The chat service reads that, then forwards to exactly those gateways.
Phase 6: Trade-offs and failure modes (2 minutes)#
A gateway crashes
Its ~50,000 clients reconnect to other gateways, with randomised delays so they don't arrive in one wave, and sync from their last seq. Nothing is lost, because a message is stored before the sender's tick. Stale registry entries expire with their TTL.
Deploying gateways
Restarting 200 stateful servers at once reconnects 10M clients at once: a thundering herd. Drain a few gateways at a time: stop accepting new connections, ask clients to reconnect elsewhere with jitter, then restart.
A hot group
A 500-member group in a busy hour is a hot partition in the message store and a burst of deliveries. Rate-limit per group, batch deliveries per gateway, and bucket long conversations by time so one partition doesn't grow forever.
End-to-end encryption
With E2E (as in the Signal protocol), the server stores and routes ciphertext it can't read. Routing, ordering and receipts work the same, but server-side search, spam scanning of content and rich push previews don't; each device must also manage keys, which complicates adding a new device.
Multi-region
Users connect to their nearest region. A conversation needs one place that assigns its seq, typically the conversation's home region; cross-region messages pay one extra hop.
Phase 7: Go deeper in the course#
- A full messaging system at WhatsApp scale: Side quest: WhatsApp
- WebSockets, long-lived connections, layer-4 load balancing: Week 0 · Part 4: Networking Fundamentals
- The session registry: Week 1 · Day 5: Session Store Design
- Client-generated IDs and safe retries: Week 2 · Day 2: Idempotency in Practice
- Ordering guarantees and what "consistent" means here: Week 5 · Day 1: Consistency Models
- Offline users and push notifications: Week 6 · Day 2: Core Notification Flow
- Reconnect storms: Week 4 · Day 3: Thundering Herd
- Rolling out stateful servers: Week 10 · Day 3: Deployment Strategies
Final check#
Final check · Chat system
Q1/3 10M concurrent connections at about 50,000 per gateway. How many gateway servers is that, before headroom?
Answer & explanation
Answer: B
10,000,000 ÷ 50,000 = 200. Add headroom so a crashed gateway's clients and rolling deploys fit on the rest.
Q2/3 What does the single "sent" tick promise Alice?
Answer & explanation
Answer: C
The ack comes after the write. "Delivered" needs a receipt from a recipient device, and "read" needs the read receipt.
Q3/3 Bob's phone was offline for an hour. How does it catch up when it reconnects?
Answer & explanation
Answer: B
Each device tracks its last sequence (or a sync cursor). One sync call returns what it missed, already in order.
New lessons by email
New lessons, side quests and case files as they ship, at most once a week. No spam, unsubscribe anytime.
Discussion · …
Public: everyone reading this lesson sees comments. Be kind and keep it on the material.