Discord.

How Do You Tell Seventy Thousand People Something at Once, and Let Any of Them Start Talking?

It is a Sunday night in October 2026 in Pune. Ishaan Kulkarni, 24, is the head moderator of Lobby 91, a Discord server for Indian esports fans with four lakh members. Tonight is the grand final of the server's own BGMI tournament. By 8:45 pm the member list on the right of his screen reads "Online — 71,204". Sixteen team voice channels sit under a "TEAMS" category, each showing the four avatars inside it; the casters are live in the stage channel; and #live-chat is scrolling faster than anyone can read.

At 8:55 he posts in #announcements: "Grand final starts in 5 minutes. Casters are live on stage. @everyone". Within a second the message is on seventy thousand screens, and phones start buzzing for hundreds of thousands of members who aren't even online. Two minutes later #live-chat fills with fresh accounts posting the same "free UC" link. Ishaan turns slowmode on, pauses invites and bans the accounts. Then a team captain messages him: the other team's player is using a banned device. Ishaan clicks "Team Lobby 3", the voice channel goes green, and he is talking to four people in Hyderabad before he has finished settling his headset.

Nothing Ishaan did that night needed a meeting link, a call to answer or a refresh. The server, its channels, its member list and its voice rooms were simply there, live, for everyone at once.

This is the system we'll design today.

The Interview Begins#

You're interviewing at a company that builds community software. The interviewer shares a screenshot: a column of server icons on the far left, a channel list with a speaker icon next to "Team Lobby 3" and four small avatars under it, a chat scrolling in the middle, and on the right a member list headed "Online — 71,204".

Your Questions#

Functional Requirements#

Non-Functional Requirements#

Connections#

Messages, Fan-Out and Storage#

One Huge Server and Presence#

Voice#

Key Metrics Summary#

System Architecture#

%% title: Discord's four planes: connections, guild state, storage, media
flowchart TD
  subgraph L1["Layer 1 · Clients"]
    direction LR
    APP@{ icon: "sd:desktop", label: "Clients<br/>desktop, phone, web, console" }
  end
  subgraph L2["Layer 2 · Edge"]
    direction LR
    GW@{ icon: "tech:websocket", label: "Gateway · Elixir<br/>websockets, zstd-stream" }
    API@{ icon: "sd:api", label: "API · Python<br/>REST, rate limits" }
  end
  subgraph L3["Layer 3 · Real-time state (Elixir)"]
    direction LR
    SESS["Sessions<br/>one process per device"]:::service
    GUILD["Guild processes<br/>one per server, relays"]:::hot
    PRES["Presence<br/>status, activities"]:::core
  end
  subgraph L4["Layer 4 · Voice"]
    direction LR
    SYNC["Voice syncers<br/>place calls on SFUs"]:::core
    SFU@{ icon: "tech:webrtc", label: "SFU fleet<br/>25,000+ instances, UDP" }
  end
  subgraph L5["Layer 5 · Storage and support"]
    direction LR
    DS["Data services · Rust<br/>coalesce by channel"]:::service
    DB@{ icon: "sd:database", label: "ScyllaDB<br/>messages, 72 nodes" }
    ETCD@{ icon: "tech:etcd", label: "etcd<br/>hash ring, discovery" }
    PUSH@{ icon: "tech:firebase", label: "Push · FCM<br/>buffer, shed bursts" }
  end
  APP ==> GW
  APP ==> API
  APP -.->|"UDP media"| SFU
  GW <==> SESS
  API ==> DS ==> DB
  API ==>|"dispatch, gRPC"| GUILD
  GUILD ==>|"fan-out"| SESS
  PRES --> SESS
  GUILD --> SYNC --> SFU
  GUILD -.-> PUSH
  ETCD -.-> GUILD
  class APP client
  class GW,API edge
  class SFU external
  class DB store
  class ETCD store
  class PUSH queue

KEY INSIGHTS:

• The guild is the unit of everything. One process per server is the single place that knows who is in it, who is online, what each role may see and who is in which voice channel; it orders that server's events and fans them out
• In over HTTP, out over the websocket. A message is stored by the API before anyone is told; the websocket only carries events down, so a dropped connection never loses a message, only the live copy of it
• The media plane is separate. Voice and video never touch the gateway; the guild only passes voice states to the voice control plane (voice syncers, in Discord's 2026 description), which picks the SFU that hosts a call, and clients are told where to connect

Data Flow: Ishaan's Announcement#

%% title: Ishaan's @everyone, from send to 71,204 screens
sequenceDiagram
  autonumber
  participant I as Ishaan's app
  participant A as API
  participant D as Data service
  participant S as ScyllaDB
  participant G as Guild process
  participant R as Relays and sessions
  participant M as Members' apps
  I->>A: POST message to #announcements, nonce n7, @everyone
  A->>A: rate limit, MENTION_EVERYONE permission
  A->>D: insert, routed by channel id
  D->>S: write to (channel, bucket), Snowflake id
  S-->>A: stored
  A-->>I: 200 with message id
  A->>G: dispatch MESSAGE_CREATE over gRPC
  G->>R: one send per relay node
  R->>R: who can see #announcements, active sessions only
  R->>M: websocket dispatch with the next sequence number
  Note over G,M: passive sessions catch up in their next periodic passive update

MEANWHILE, IN PARALLEL:

• The guild hands the @everyone mention to the push pipeline for members who aren't active in the app; the pipeline buffers and, in a catastrophic burst, sheds
• Thousands of members open #announcements; their history requests carry the channel id as a routing key, land on the same data-service instance, and share one ScyllaDB query

Deep Dive 1: One Process per Server, and the Server With a Million Online (Week 1 Concepts: Hot Keys and Skew, Partitioning)#

This builds on Week 1, Day 4 · Hot Keys and Skew and Week 1, Day 1 · Partitioning Deep Dive.

What is published: Discord runs "a single Elixir process per guild as a central routing point for everything happening on that server, and another process (a 'session') for each connected user's client", and keeps one guild process because "some things need a full view", such as who is in a voice channel. In 2017, with servers like /r/Overwatch at up to 30,000 concurrent users, it built Manifold, which sends one copy per remote node and lets that node fan out across its cores. In 2023 it described passive sessions (about 90% of connections in large servers, making fan-out "90% less expensive"), relays that own up to 15,000 sessions each and do "fanout with permission checks", a member list in ETS so worker processes can do slow jobs like working out who can see an @everyone, and Manifold Offload, a separate sender process that only paid off after tuning garbage collection of a heap "gigabytes in size".

%% title: A huge guild: one process orders, relays fan out
flowchart TD
  API@{ icon: "sd:api", label: "API<br/>MESSAGE_CREATE via gRPC" }
  G["Guild process<br/>order, roles, voice states"]:::hot
  ETS@{ icon: "sd:database", label: "ETS member table<br/>shared, read by workers" }
  W["Worker process<br/>who can see @everyone"]:::service
  R1["Relay 1<br/>15,000 sessions"]:::core
  R2["Relay 67<br/>15,000 sessions"]:::core
  ACT@{ icon: "sd:users", label: "Active sessions<br/>every event" }
  PAS@{ icon: "sd:users", label: "Passive sessions<br/>periodic deltas" }
  API ==> G
  G -.-> ETS
  ETS -.-> W
  G ==>|"one send per relay"| R1
  G ==> R2
  R1 ==>|"permission-checked"| ACT
  R2 -.->|"summary"| PAS
  class API edge
  class ETS store
  class ACT,PAS client

• Place guilds by consistent hashing. A guild id maps to a node on a hash ring. Discord found the lookup itself was a hot path (500,000 sessions × 5 guilds × 12 µs ≈ 30 seconds after a restart) and compiled the ring into a constant (FastGlobal): about 0.3 µs a lookup.
• Keep small guilds simple. A server of 20 people gets no relays: the guild talks to its sessions directly. Relays start above a size threshold (ours: ~5,000 connected sessions) and are added as sessions arrive, each up to 15,000.
• Give relays only what they need. Copying the whole member list into every relay meant dozens of copies of tens of millions of members, and stalled the guild "for tens of seconds" per new relay; Discord now sends each relay "a tiny percentage of the overall members".
• Move slow work off the loop. Anything that walks all members runs in a worker reading ETS. Discord uses this to hand a guild between machines during deploys, sending "multiple GB" of members while the old process keeps working.
• Quadratic means diminishing returns. A 10× cut in work buys about a 3× bigger server. Expect to keep finding the next bottleneck; budget for it.

# guilds/fanout.py

"""
Fan-out from one guild (illustrative). The guild orders events; relays own sessions,
do the permission checks and the sends. Small guilds talk to sessions directly.
Passive sessions get nothing here: they receive periodic deltas instead.
"""

RELAY_THRESHOLD = 5_000      # connected sessions before a guild starts using relays
RELAY_CAPACITY = 15_000      # Discord: up to 15,000 sessions per relay


class Guild:
    def __init__(self, gid, members, roles, channels):
        self.gid, self.members, self.roles, self.channels = gid, members, roles, channels
        self.seq = 0
        self.direct = {}            # session_id -> session process (small guilds)
        self.relays = []            # relay processes, each with up to RELAY_CAPACITY sessions

    def connect(self, session):
        if not self.relays and len(self.direct) < RELAY_THRESHOLD:
            self.direct[session.id] = session
            return
        if not self.relays:
            self.relays = [spawn_relay(self, list(self.direct.values()))]
            self.direct.clear()
        relay = min(self.relays, key=lambda r: r.size)
        if relay.size >= RELAY_CAPACITY:
            relay = spawn_relay(self, [])                      # only the members it needs
            self.relays.append(relay)
        relay.add(session)

    def dispatch(self, event):
        self.seq += 1                                          # one order for the whole guild
        event = {**event, "guild_seq": self.seq}
        if self.relays:
            for relay in self.relays:                          # ~67 sends for a million sessions
                relay.send(event)
            return
        for s in self.direct.values():
            if s.active and can_see(self, s.user_id, event["channel_id"]):
                s.send(event)

Deep Dive 2: The Gateway: Millions of Sockets, Fan-Out and Backpressure (Week 3 Concepts: Backpressure and Flow Control; Week 4: Thundering Herd)#

This builds on Week 3, Day 3 · Backpressure and Flow Control and Week 4, Day 3 · Thundering Herd.

What is published: the gateway is "both the ingress and egress point for all websocket traffic". Clients wait heartbeat_interval * jitter before the first heartbeat (45,000 ms in the docs' example) so clients don't all reconnect "at the exact same time"; every dispatch carries a sequence number, and a client that resumes "will send the missed events in order". Streaming zstandard (2024) cut a MESSAGE_CREATE from 270 to 166 bytes, and delta updates for passive sessions cut them from 35% to 5% of gateway bandwidth: "almost 40%" less in all. Discord notes autoscaling "doesn't work well" for long-lived connections. In 2017, 5 million sessions stampeding ten guild-registry processes caused cascading out-of-memory crashes; its Semaphore library, an atomic ETS counter, now makes callers fail fast. Pushes go through GenStage: pushers demand at most 100 requests, Firebase's limit per connection, and the collector's buffer sheds in "catastrophic situations".

%% title: A connection's life: identify, heartbeat, resume
stateDiagram-v2
  [*] --> CONNECTING: open websocket
  CONNECTING --> IDENTIFYING: Hello, heartbeat interval
  IDENTIFYING --> READY: session started, READY payload
  READY --> READY: dispatch with sequence n
  READY --> RESUMING: socket dropped or buffer overflow
  RESUMING --> READY: missed events replayed, Resumed
  RESUMING --> IDENTIFYING: session gone, invalid session
  READY --> CLOSED: user quits
  CLOSED --> [*]
  class READY ok
  class RESUMING queue

• The session is not the socket. A socket can die in a tunnel; the session process (and its replay buffer) lives on for a short window. Discord's gateway monitors each session and, if the session is still alive, "we hook everything back up". Losing a socket costs a resume; losing a session costs a full READY.
• READY is the expensive packet. Discord's example READY was 2.5 MB before compression. Make it lazy (guild stubs first, member lists only for what's on screen), so a reconnect storm doesn't become a bandwidth storm.
• Bound every queue. Each session has an outbound buffer cap (ours: ~5,000 events or ~2 MB); overflow closes the socket with a resumable code. Each guild has a mailbox alarm; when it crosses a threshold, it sheds what can be rebuilt (typing indicators, presence for passive sessions) before what can't (messages).
• Fail fast on overload. A session that can't get a semaphore slot for a busy service errors at once and retries with backoff, rather than joining a queue it will time out of.
• Compress per connection. One streaming context per socket learns that connection's payloads, which are mostly a few hundred bytes; it costs memory per connection, so size it to the node.
• Pushes are allowed to drop; messages aren't. A missed push is a missed buzz; the message is still in the channel. That's why the push pipeline may shed under a burst while the message path never does.

# gateway/session_buffer.py

"""
A session's outbound side (illustrative). Every dispatch gets a sequence number and
is kept in a bounded replay buffer; a slow socket overflows into a resumable close,
not into unbounded memory. Resume replays only what the client missed.
"""

from collections import deque

MAX_PENDING = 5_000          # events queued for a socket that isn't reading
REPLAY_WINDOW = 10_000       # events kept for resume


class Session:
    def __init__(self, session_id):
        self.id, self.seq = session_id, 0
        self.replay = deque(maxlen=REPLAY_WINDOW)
        self.pending = deque()
        self.socket = None

    def dispatch(self, event):
        self.seq += 1
        frame = {"op": 0, "s": self.seq, "t": event["type"], "d": event["data"]}
        self.replay.append(frame)
        if self.socket is None:
            return                                        # disconnected: replay buffer only
        if len(self.pending) >= MAX_PENDING:
            self.socket.close(code=4000)                  # illustrative: Discord's docs list 4000 as
                                                          # "Unknown error", reconnect allowed
            self.socket = None
            self.pending.clear()
            return
        self.pending.append(frame)

    def resume(self, socket, last_seq):
        oldest = self.replay[0]["s"] if self.replay else self.seq + 1
        if last_seq + 1 < oldest:
            return "INVALID_SESSION"                      # missed too much: identify again
        self.socket = socket
        for frame in self.replay:
            if frame["s"] > last_seq:
                self.pending.append(frame)                # in order, then a RESUMED marker
        self.pending.append({"op": 0, "t": "RESUMED", "s": self.seq, "d": {}})
        return "RESUMED"

Deep Dive 3: Presence and the Member List (Week 5 Concepts: Consistency Models)#

This builds on Week 5, Day 1 · Consistency Models and Week 1, Day 5 · Session Store Design.

What is published: in 2019 Discord described sending member-list updates only for "the visible portion", which needed a sorted set reporting the index of every change; pure Elixir topped out around 250,000 members, so it wrote SortedSet in Rust (0.61 to 3.68 µs per insert at up to 1,000,000 items), which "powers every single Discord guild". The gateway docs show the same idea for apps: above large_threshold (50 to 250) members, offline members are left out, and for guilds over 75,000 members only members in voice are sent until requested. When a session exits, process monitors tell its guilds to "stop trying to send you messages, presence updates, and everything else". A 2026 post describes a presence service "which fans out DMs and status updates to users".

%% title: One idle event, delivered only where it's visible
flowchart TD
  S1@{ icon: "sd:desktop", label: "Desktop session<br/>online, Playing BGMI" }
  S2@{ icon: "sd:phone", label: "Phone session<br/>idle 10 min" }
  P["Presence<br/>combine sessions, coalesce"]:::core
  G["Guild process<br/>member list, SortedSet"]:::hot
  F@{ icon: "sd:users", label: "Friends<br/>status change" }
  V@{ icon: "sd:users", label: "Sessions viewing rows 0-99<br/>update at index 37" }
  N@{ icon: "sd:users", label: "Everyone else<br/>nothing now, count later" }
  S1 --> P
  S2 --> P
  P ==> G
  P --> F
  G ==> V
  G -.-> N
  class S1,S2 client
  class F,V,N client

• Session status, user presence. Store status per session; derive the user's presence from all live sessions. One device going idle changes nothing visible if another is active, so many changes stop at this step.
• Coalesce. If a user goes online, idle and online again within a few seconds (a phone switching networks), send the last state only. A presence change can wait a second or two; a message can't.
• Member list as ranges. The guild keeps the sorted member list (role, status, name) and, per session, the ranges it watches. A change becomes an operation at an index, sent only to sessions watching that index. Very large guilds can show only online members, so offline members never enter the list at all.
• Consistency by kind. Messages: per-channel order, read-your-writes for the sender. Presence and counts: eventual, seconds. Permissions: strong, because a role removed from a member must stop the next message reaching them, so permission changes go through the guild's ordered loop like messages.
• Read states are their own service. Unread counts and mentions are one record per user per channel, "billions" of them; Discord keeps them in LRU caches committed to Cassandra, and rewrote the service from Go to Rust in 2020 to remove garbage-collection spikes.

# presence/aggregate.py

"""
User presence from session statuses, and member-list updates only for watchers
(illustrative). Offline comes from a session exiting (a process monitor firing),
never from the client announcing it.
"""

RANK = {"online": 3, "dnd": 3, "idle": 2}


def user_presence(sessions: dict) -> dict:
    """sessions: session_id -> {"status": ..., "activities": [...], "since": t}"""
    live = [s for s in sessions.values() if s["status"] != "offline"]
    if not live or any(s["status"] == "invisible" for s in live):
        return {"status": "offline", "activities": []}       # invisible looks offline to others
    best = max(live, key=lambda s: (RANK[s["status"]], s["since"]))
    return {"status": best["status"], "activities": [a for s in live for a in s["activities"]]}


def on_member_change(member_list, watchers, user_id, new_presence):
    """member_list: SortedSet-like with index_of/move; watchers: session -> [(lo, hi)]"""
    old_i = member_list.index_of(user_id)
    new_i = member_list.update(user_id, new_presence)        # re-sorts by role, status, name
    ops = [("UPDATE", new_i)] if old_i == new_i else [("DELETE", old_i), ("INSERT", new_i)]
    for session, ranges in watchers.items():
        touched = [(op, i) for op, i in ops if any(lo <= i <= hi for lo, hi in ranges)]
        if touched:
            # "MEMBER_LIST_UPDATE" is our illustrative event name; it is not in Discord's public docs
            session.send({"t": "MEMBER_LIST_UPDATE", "ops": touched, "presence": new_presence})

Deep Dive 4: Trillions of Messages: Hot Partitions and a Days-Not-Months Migration (Week 1 Concepts: Partitioning, Hot Keys; Week 4: Thundering Herd)#

This builds on Week 1, Day 1 · Partitioning Deep Dive, Week 1, Day 4 · Hot Keys and Skew and Week 4, Day 3 · Thundering Herd. The migration itself is the war story Discord moved trillions of messages to a new database in days, not months.

What is published: Discord moved from MongoDB to Cassandra in 2015–2016 with the key ((channel_id, bucket), message_id): a Snowflake id (time-sortable) and a bucket of about 10 days to keep partitions under 100 MB. A channel where millions of messages had been deleted took 20 seconds to load because of tombstones; Discord cut their lifetime from 10 days to 2 and stopped writing nulls, which had created 12 tombstones per message. By 2022, 177 Cassandra nodes held trillions of messages, and hot partitions raised latency "across our entire database cluster". Discord put Rust data services in front, "roughly one gRPC endpoint per database query" whose "big feature … is request coalescing", routed by channel id; then dual-wrote to ScyllaDB, copied history with a Rust migrator at up to 3.2 million messages a second (its estimate was nine days; "several days later" it reached 99.9999% and stalled on tombstones until they were compacted away), compared a sample of reads, and switched in May 2022. Result: 72 nodes instead of 177, p99 reads from 40–125 ms to 15 ms, inserts from 5–70 ms to 5 ms.

%% title: Reading the latest messages: route, coalesce, then one query
flowchart TD
  C@{ icon: "sd:users", label: "20,000 clients<br/>latest 50 in #announcements" }
  API@{ icon: "sd:api", label: "API<br/>routing key = channel id" }
  DS["Data service instance<br/>one per channel by hash"]:::hot
  T["In-flight task<br/>channel 81, bucket 412"]:::core
  DB@{ icon: "sd:database", label: "ScyllaDB<br/>(channel, bucket) partition" }
  C ==> API
  API ==>|"consistent hash"| DS
  DS ==>|"first request starts it"| T
  DS -.->|"19,999 subscribe"| T
  T ==>|"one query"| DB
  class C client
  class API edge
  class DB store

• Snowflake ids are the clustering key. They're unique and sort by time, so "messages before id X" is a range scan, and a message id tells you its bucket without a lookup. Discord's ids carry 42 bits of milliseconds since the first second of 2015.
• Buckets bound partitions. Bucket = (timestamp since the epoch) ÷ ~10 days. To load recent messages, scan buckets from now backwards until you have 50; quiet channels touch a few, busy channels one.
• Coalescing beats caching here. A cache needs invalidation on every edit, delete and reaction; coalescing just shares the result of a query already in flight, so it can't serve anything stale beyond that one query. Discord said this cut traffic spikes like an @everyone announcement "significantly".
• Route by the thing that's hot. Coalescing only works if identical requests meet on the same instance, so the routing key is the channel id; a hot channel concentrates on one data-service instance, which is cheap, instead of on the database, which isn't.
• Migrate in steps, newest first. Move the smaller databases first and learn the new one in production; put the protecting layer in front; dual-write new data; copy history behind it with checkpoints (Discord's migrator checkpointed token ranges locally in SQLite); compare a sample of reads; then flip. Each step is reversible until the last.
• Search shards by guild too, and the biggest guilds hit Lucene's ~2 billion document limit; Discord now gives them a dedicated Elasticsearch cell with multi-shard indices (Discord Engineering, April 2025).

# data/messages.py

"""
Message reads behind a data service (illustrative): Snowflake ids give the bucket,
requests for the same channel are routed to one instance, and identical in-flight
queries are coalesced so a stampede costs one database read.
"""

import asyncio

DISCORD_EPOCH_MS = 1_420_070_400_000          # first second of 2015
BUCKET_MS = 10 * 24 * 3600 * 1000             # ~10 days per partition


def bucket_of(snowflake: int) -> int:
    return (snowflake >> 22) // BUCKET_MS     # top 42 bits: ms since the Discord epoch


class Coalescer:
    def __init__(self, db):
        self.db, self.inflight = db, {}

    async def latest(self, channel_id: int, before: int, limit: int = 50):
        key = (channel_id, before, limit)
        if key in self.inflight:
            return await self.inflight[key]               # share the query already running
        task = asyncio.ensure_future(self._read(channel_id, before, limit))
        self.inflight[key] = task
        try:
            return await task
        finally:
            self.inflight.pop(key, None)

    async def _read(self, channel_id, before, limit):
        out, b = [], bucket_of(before)
        floor = bucket_of(channel_id)                     # a channel id is a Snowflake too
        while len(out) < limit and b >= floor:
            if not await self.db.bucket_known_empty(channel_id, b):
                out += await self.db.select(channel_id, b, before=before, limit=limit - len(out))
            b -= 1
        return out

Deep Dive 5: Voice Channels You Can Drop Into (Week 10 Concepts: Capacity Planning; Week 9: Security Architecture)#

This builds on Week 10, Day 4 · Capacity Planning and Week 9, Day 5 · Security Architecture. For how an SFU forwards rather than mixes, and simulcast, see Bonus Problem 23 (Zoom).

What is published: in 2018 a voice server was a signalling component plus a home-grown C++ SFU; servers reported load into etcd, and the guild service picked "the least utilized voice server in the given region" for the whole guild: all of a guild's voice channels shared one voice server. Clients show "Awaiting Endpoint" while one is chosen and "Voice Connected" once UDP works. A crashed SFU is restarted and rebuilt "without any client interaction"; a dead or DDoS'd voice server is removed from discovery and its calls reassigned. In 2026 Discord described voice syncers that "decide where to place and manage every active call" over "more than 25,000 external" SFU instances, with one syncer per guild and child syncers per voice channel; "we pick one SFU instance to host an entire call"; a newer Rust-based SFU; and 80%+ of traffic on Cloudflare's edge. DAVE encrypts each frame with per-sender keys from an MLS group that the SFU can't read; since early March 2026 it covers every call except stage channels.

%% title: Joining a voice channel: control over the gateway, media over UDP
sequenceDiagram
  autonumber
  participant C as Ishaan's client
  participant GW as Gateway
  participant G as Guild process
  participant VS as Voice syncer
  participant V as Voice server
  C->>GW: voice state: join Team Lobby 3
  GW->>G: voice state update
  G->>VS: channel has 5 voice states
  Note over VS: call already placed on a host near its members
  VS-->>G: endpoint and token
  G-->>C: voice server update, endpoint and token
  C->>V: voice websocket identify, token, max DAVE version
  V-->>C: ready, SSRC, UDP address, encryption modes
  C->>V: UDP IP discovery, then select protocol
  V-->>C: session description, mode, DAVE version
  C->>V: MLS key package
  V-->>C: MLS welcome, new epoch
  Note over C,V: Voice Connected · encrypted Opus frames flow

• The room exists before you enter. A voice channel with people in it already has a host; joining is a voice-state write plus a websocket and a UDP handshake to a known address. Placement happens once, when the first person joins, so "Awaiting Endpoint" should be rare and short.
• Place by where people are, not where the first person is. In Discord's first Cloudflare test, in Iceland in February 2025, a call started by an Icelander with friends in Germany could land in Reykjavik, and ping for non-Icelanders on Iceland-hosted calls "jumped 2.7x". A call has one host; choose it from the participants' regions (and re-evaluate when the mix changes), not from the first joiner alone.
• Silence is free. No speech, no packets; the SFU forwards each speaker's stream to each listener, so a big channel with one talker costs one stream per listener. That's why Discord's 2018 totals work out to only ~85 kbit/s per voice user (our arithmetic: 220 Gbps ÷ 2.6 million), and why a channel where 1,000 people take turns speaking is affordable.
• Moderation lives in the SFU. Server mute drops a user's packets at the SFU; a moderator "moving" someone is a voice-state change the guild orders. With E2EE, the SFU still forwards and drops packets, but can't read them.
• The control plane is the fragile part. SFUs forward packets independently; what can collapse is the service that creates calls and talks to thousands of SFUs. Rate-limit call creation per guild and per channel, and partition the control plane so one overloaded process can't block the rest (see the Final Challenge).
• Capacity is per host and per region. Four SFU workers sharing one NIC queue lost 1.5 to 2% of packets at peak, so Discord halved density until the platform was fixed. Plan voice capacity with packet loss, not CPU, as the limit.

# voice/placement.py

"""
Choosing a host for a call (illustrative). One SFU hosts each call; pick the region
that minimises the participants' worst latency, then the least-loaded healthy host
there. Moving a call reconnects everyone, so re-place only for a large gain.
"""

def choose_region(participants, rtt_ms) -> str:
    """participants: list of client regions; rtt_ms[(client_region, host_region)] -> ms"""
    hosts = {h for (_, h) in rtt_ms}
    return min(hosts, key=lambda h: (max(rtt_ms[(p, h)] for p in participants),
                                     sum(rtt_ms[(p, h)] for p in participants)))


def choose_host(discovery, region, exclude=()):
    healthy = [h for h in discovery.hosts(region)
               if h.healthy and not h.under_attack and h.id not in exclude]
    if not healthy:
        raise NoCapacity(region)
    return min(healthy, key=lambda h: h.load)              # least utilised, as in 2018

Deep Dive 6: Rate Limits, Spam and Raids (Week 1 Concepts: Rate Limiting at Scale; Week 9: Noisy Neighbours)#

This builds on Week 1, Day 3 · Rate Limiting at Scale and Week 9, Day 2 · Noisy Neighbour.

What is published: Discord's API has per-route limits that "account for top-level resources" (channel, guild or webhook ids); a global limit of 50 requests a second per bot; an invalid-request limit of 10,000 per 10 minutes per IP, after which the IP is temporarily banned; and X-RateLimit-* headers including a Scope of user, global or shared, with shared 429s "not counted against you". On the gateway, an app may send 120 events per 60 seconds per connection and IDENTIFY at most 1,000 times in 24 hours (more for very large bots). Slowmode runs from 0 to 21,600 seconds and doesn't apply to bots or to users with the bypass-slowmode permission. In September 2022 Discord said AutoMod had removed "over 20 million unwanted messages" since June, and added spam-content and mention-spam blocking, paused invites and raid reports. Internally, the gateway has a start-session rate limit for backpressure, and a 2025 project added automatic circuit breaking to shed demand in the API when it is overloaded (Discord Engineering, April 2026).

%% title: One message, five checks before it is fanned out
flowchart TD
  R@{ icon: "sd:api", label: "POST message<br/>bot token, channel 81" }
  GL{"Global per-bot limit<br/>50 a second"}
  RT{"Route bucket<br/>route + channel 81"}
  SM{"Slowmode<br/>users only, bots exempt"}
  AM{"AutoMod<br/>spam link, mention spam"}
  OK["Store and dispatch"]:::ok
  NO["429 with retry_after<br/>bucket and scope"]:::bad
  BLK["Blocked before posting<br/>moderators alerted"]:::bad
  R ==> GL
  GL ==>|"under"| RT
  GL -.->|"over"| NO
  RT ==>|"under"| SM
  RT -.->|"over"| NO
  SM ==>|"allowed"| AM
  SM -.->|"too soon"| NO
  AM ==>|"clean"| OK
  AM -.->|"match"| BLK
  class R edge

• Key limits by the resource that costs. A route bucket per channel means a bot busy in one server can't exhaust its quota in another, and one channel's spam can't slow other channels. The cost of a message is set by where it goes, so the expensive cases get their own limits: @everyone needs a permission, and AutoMod can block messages that mention too many members.
• Return the limits, don't make clients guess. Remaining and reset headers let a well-behaved client slow down before it's refused. Scope tells it whether the limit was its own (back off), global (back off everything) or shared (someone else filled the resource's bucket: wait, it isn't counted).
• Punish loops, not bursts. The invalid-request limit targets clients that keep sending requests they know will fail. A bot that hits 429 once and waits is fine; one that retries in a loop is cut off at the edge before it reaches any service.
• Limit work, not just requests. The worst floods are internal: session starts after a mass disconnect, calls being recreated. Each gets its own limit, re-tuned after every infrastructure change (see the Final Challenge).
• Fan-out is a noisy-neighbour problem. One busy server shares nodes with thousands of small ones; relays, mailbox alarms and shedding typing and presence first keep its tournament night from becoming everyone's lag.

# ratelimit/buckets.py

"""
Per-route, per-resource token buckets with Discord-style response headers
(illustrative). The bucket key includes the top-level resource (channel, guild or
webhook id), so one channel's limit never blocks another.
"""

import time
import re

TOP_LEVEL = re.compile(r"/(channels|guilds|webhooks)/(\d+)")


class Bucket:
    def __init__(self, limit: int, per_s: float):
        self.limit, self.per_s = limit, per_s
        self.tokens, self.reset_at = limit, time.time() + per_s

    def take(self, now: float):
        if now >= self.reset_at:
            self.tokens, self.reset_at = self.limit, now + self.per_s
        if self.tokens == 0:
            return False
        self.tokens -= 1
        return True


def check(store, principal: str, method: str, route: str, path: str, policy: dict):
    now = time.time()
    if not store.bucket(f"global:{principal}", 50, 1.0).take(now):
        return 429, {"X-RateLimit-Global": "true", "X-RateLimit-Scope": "global"}
    m = TOP_LEVEL.search(path)
    resource = m.group(2) if m else "-"
    limit, per_s = policy[(method, route)]
    b = store.bucket(f"{principal}:{method}:{route}:{resource}", limit, per_s)
    ok = b.take(now)
    headers = {"X-RateLimit-Limit": str(limit), "X-RateLimit-Remaining": str(b.tokens),
               "X-RateLimit-Bucket": f"{method}:{route}",
               "X-RateLimit-Reset-After": f"{b.reset_at - now:.3f}"}
    if not ok:
        store.count_invalid(principal)                     # 10,000 in 10 min → edge ban
        return 429, {**headers, "X-RateLimit-Scope": "user", "Retry-After": f"{b.reset_at - now:.3f}"}
    return 200, headers

Ishaan's Tournament Night#

Edge Cases#

What is published: Discord's "guild timings" record "how much of the current minute has been spent on each action type". In 2026 it added tracing from the API's dispatch (1.69 ms in its example) through the guild's fan-out (357 µs) into each session, where each session keeps a sampled trace's spans 100% of the time for a single-recipient message, 10% at 100 recipients and 0.1% at 10,000+ (on top of head sampling), so a million-session fan-out doesn't create a million spans. Voice is measured by client-reported ping, packet loss and "expand ratio (the measurement of audio frames we stretch to cover gaps)". After its March 2026 outage, Discord added monitoring of its HTTP connection-pool mailboxes and is expanding monitoring of its most critical processes.

Monitoring Dashboard#

SLOs for the Platform#

The interviewer leans back. "This one's from Discord's own postmortem. Tell me what you'd have done, and what you'd change."

Summary: Concepts Applied from 10-Week Course#

Why Discord Matters#

Self-Assessment Checklist#

After studying this case study, you should be able to:

Fan-out and connections:

Presence and storage:

Voice and limits:

Sources#

Discord Engineering (discord.com/blog):

• Blog

  Yuliy Pisetsky — Maxjourney: Pushing Discord's Limits with a Million+ Online Users in a Single Server (guild processes; quadratic fan-out; Midjourney 10M+ members, 1M+ online; passive sessions; relays of up to 15,000 sessions; ETS and workers; Manifold Offload), 25 October 2023: https://discord.com/blog/maxjourney-pushing-discords-limits-with-a-million-plus-online-users-in-a-single-server

• Blog

  Stanislav Vishnevskiy — How Discord Scaled Elixir to 5,000,000 Concurrent Users (Manifold; consistent hashing and FastGlobal; 500,000 sessions per VM; 5 guilds per user; Semaphore), 6 July 2017: https://discord.com/blog/how-discord-scaled-elixir-to-5-000-000-concurrent-users

• Blog

  Matt Nowack — Using Rust to Scale Elixir for 11 Million Concurrent Users (visible-portion member lists; Rust SortedSet), 17 May 2019: https://discord.com/blog/using-rust-to-scale-elixir-for-11-million-concurrent-users

• Book

  Austin Whyte — How Discord Reduced Websocket Traffic by 40% (streaming zstandard; READY size; passive session deltas; gateway autoscaling), 20 September 2024: https://discord.com/blog/how-discord-reduced-websocket-traffic-by-40-percent

• Blog

  Jesse Howarth — How Discord handles push request bursts of over a million per minute with Elixir's GenStage (demand of 100; buffer and shedding), 12 December 2016: https://discord.com/blog/how-discord-handles-push-request-bursts-of-over-a-million-per-minute-with-elixirs-genstage

• Blog

  Stanislav Vishnevskiy — How Discord Stores Billions of Messages (history forever; 120M+ messages a day; 50/50 reads and writes; 10-day buckets; Snowflakes; tombstones), 13 January 2017: https://discord.com/blog/how-discord-stores-billions-of-messages

• Blog

  Bo Ingram — How Discord Stores Trillions of Messages (177 Cassandra nodes; data services and coalescing; ScyllaDB migration estimated at 9 days, at up to 3.2M a second; 72 nodes × 9 TB; latencies), 6 March 2023: https://discord.com/blog/how-discord-stores-trillions-of-messages

• Blog

  Jesse Howarth — Why Discord is switching from Go to Rust (Read States service), 4 February 2020: https://discord.com/blog/why-discord-is-switching-from-go-to-rust

• Blog

  Vicki Niu — How Discord Indexes Trillions of Messages (Lucene's ~2 billion document limit; cells; 40 clusters), 24 April 2025: https://discord.com/blog/how-discord-indexes-trillions-of-messages

• Blog

  Jozsef Vass — How Discord Handles Two and Half Million Concurrent Voice Users using WebRTC (gateway, guilds and voice; C++ SFU; placement, failover and DDoS; 850+ servers, 13 regions; 2.6M+ users, 220+ Gbps, 120 Mpps), 10 September 2018: https://discord.com/blog/how-discord-handles-two-and-half-million-concurrent-voice-users-using-webrtc

• Talk

  Stephen Birarda — Meet DAVE: Discord's New End-to-End Encryption for Audio & Video (~200M people a month; MLS; frame encryption the SFU can't read), 17 September 2024: https://discord.com/blog/meet-dave-e2ee-for-audio-video

• Talk

  Mark Smith — Every Voice and Video Call on Discord Is Now End-to-End Encrypted (complete in early March 2026; stage channels excepted), 18 May 2026: https://discord.com/blog/every-voice-and-video-call-on-discord-is-now-end-to-end-encrypted

• Blog

  Josh Stratton — How It All Goes Live: An Overview of Discord's Streaming Technology, 5 March 2024: https://discord.com/blog/how-it-all-goes-live-an-overview-of-discords-streaming-technology

• Blog

  David Chen — How We Moved Discord Voice to the Edge (80%+ of traffic on Cloudflare; one SFU host per call; Iceland 2.7× ping; NIC queue packet loss; Rust SFU; expand ratio), 9 June 2026: https://discord.com/blog/how-we-moved-discord-voice-to-the-edge

• Blog

  Bo Ingram and Stephen Birarda — You've Got (Too Much) Mail: Behind the Scenes of the 3/25/26 Voice Outage (17% of sessions; gateway and resume; voice syncers and 25,000+ SFUs; timeline and fixes), 29 April 2026: https://discord.com/blog/behind-the-scenes-of-the-3-25-26-voice-outage

• Blog

  Nick Krichevsky — Tracing Discord's Elixir Systems (Without Melting Everything) (guild timings; recipient-scaled sampling; presence service), 4 March 2026: https://discord.com/blog/tracing-discords-elixir-systems-without-melting-everything

• Blog

  Discord — Blocking Spam Gets Easier Thanks to New AutoMod and Safety Tools (20M+ messages removed; mention spam; paused invites), 21 September 2022: https://discord.com/blog/new-anti-spam-raid-automod-safety-update

Discord company page and developer documentation:

• Read

  Discord — Company ("90M+ Daily Active Users", internal data as of Q4 2025): https://discord.com/company

• Docs

  Discord Developer Docs — Gateway (heartbeat jitter; resume; IDENTIFY and event limits; sharding; intents; compression): https://github.com/discord/discord-api-docs/blob/main/developers/events/gateway.mdx

• Docs

  Discord Developer Docs — Gateway Events (large_threshold; 75,000-member guilds; member chunks): https://github.com/discord/discord-api-docs/blob/main/developers/events/gateway-events.mdx

• Docs

  Discord Developer Docs — Rate Limits (per-route, global and invalid-request limits; headers and scopes): https://github.com/discord/discord-api-docs/blob/main/developers/topics/rate-limits.mdx

• Docs

  Discord Developer Docs — Voice Connections (voice websocket, UDP, IP discovery, DAVE): https://github.com/discord/discord-api-docs/blob/main/developers/topics/voice-connections.mdx

• Docs

  Discord Developer Docs — Channel resource (slowmode 0–21,600 s) and Reference (Snowflakes): https://github.com/discord/discord-api-docs/blob/main/developers/resources/channel.mdx · https://github.com/discord/discord-api-docs/blob/main/developers/reference.mdx

Design note: the architecture beyond what Discord's posts describe, the code, thresholds, dashboards and every number labelled as an assumption or estimate are an illustrative design, not Discord's internals. Peak concurrency, sessions per user, message and event rates, recipients per message, presence change rates, voice users today, bytes per message and all of Lobby 91's numbers are assumptions. Ishaan Kulkarni, Lobby 91 and its tournament are fictional. Discord's published numbers come from posts between 2016 and 2026 and describe the system at the time of each post.

Further Reading#

Related systems to study:

• Read

  WhatsApp (Bonus Problem 2 in this series): one-to-one and group messaging on Erlang, with offline delivery and end-to-end encryption of text

• Read

  Zoom (Bonus Problem 23): scheduled meetings, SFU forwarding, simulcast and cascading across regions

• Read

  Twitter / X (Bonus Problem 10): fan-out on write versus read for a follower graph

• Docs

  Figma (Bonus Problem 15): one authoritative process per document, the same idea as one process per guild

From the war stories:

• Read

  Discord moved trillions of messages to a new database in days, not months

Foundational concepts (this course):

• Read

  Week 1, Day 1 — Partitioning Deep Dive

• Read

  Week 1, Day 3 — Rate Limiting at Scale

• Read

  Week 1, Day 4 — Hot Keys and Skew

• Read

  Week 1, Day 5 — Session Store Design

• Read

  Week 3, Day 3 — Backpressure and Flow Control

• Read

  Week 4, Day 3 — Thundering Herd

• Read

  Week 5, Day 1 — Consistency Models

• Read

  Week 9, Day 2 — Noisy Neighbour

• Read

  Week 9, Day 5 — Security Architecture

• Read

  Week 10, Day 4 — Capacity Planning

"One server, one ordered process; push the fan-out outwards, and send each person only what they can see."