Slack design sheet · System Design Mastery

At a glance#

Number	Value	What it drives
Connected users	12.5M (Slack, March 2020)	~30M websockets today (assumed)
Delivery	500 ms worldwide (2023)	two-hop fan-out, regional gateways
Channels per channel server	~16M at peak (2023)	~50 servers at half load (assumed)
Channel server replacement	under 20 s (2023)	ring repair, resubscribe, resync
Database	2.3M QPS · p99 11 ms (2020)	Vitess keyspaces by channel and user
Jobs	1.4B on busiest days · 33,000/s peak (2017)	Kafka in front of Redis
Messages	1B+ a week (2019)	~20,000/s at a peak minute (assumed)
Shared channel	up to 250 organisations	one copy, per-side views

Architecture#

DiagramWrite path, real-time path, edge cache and async path

%% title: Write path, real-time path, edge cache and async path
flowchart TD
  APP@{ icon: "sd:laptop", label: "Slack clients<br/>desktop · mobile · web" }
  ENV@{ icon: "sd:load-balancer", label: "Envoy at the edge<br/>TLS · websockets · zone weights" }
  FL["Flannel<br/>edge cache · team affinity"]:::service
  GS["Gateway servers<br/>websockets · subscriptions"]:::core
  WEB@{ icon: "sd:api", label: "Webapp · Hack<br/>every API and write" }
  AS["Admin servers<br/>stateless routers"]:::service
  CS["Channel servers<br/>hash ring · in-memory"]:::hot
  CON@{ icon: "sd:network", label: "Consul<br/>ring from CHARMs" }
  VT@{ icon: "tech:mysql", label: "Vitess · MySQL<br/>Memcached in front" }
  KF@{ icon: "tech:kafka", label: "Kafkagate · Kafka<br/>durable job log" }
  RQ@{ icon: "tech:redis", label: "JQRelay · Redis<br/>queues · workers" }
  PUSH@{ icon: "sd:bell", label: "Push service<br/>APNs · FCM" }
  SR@{ icon: "sd:search", label: "Search · Solr<br/>per tenant · ranked" }
  APP ==> ENV
  ENV ==>|"websocket"| GS
  ENV ==>|"HTTPS"| WEB
  ENV -.->|"boot · queries"| FL
  WEB ==>|"store first"| VT
  WEB ==> AS ==> CS
  CS ==>|"one copy per gateway"| GS
  CON -.-> CS
  WEB ==>|"jobs"| KF ==> RQ
  RQ --> PUSH
  RQ --> SR
  class APP client
  class ENV,WEB edge
  class VT,CON store
  class KF,RQ queue
  class PUSH external
  class SR service

Store, then send, then do the rest: the write lands in Vitess before fan-out; notifications, unfurls and indexing are jobs.
Route by channel: every event is addressed to a channel ID; one company's channels spread over every channel server.
Boot cheap at the edge: Flannel serves slim boots and queries from a per-workspace cache in each region.

DiagramZoom-in · two hops of fan-out

%% title: Zoom-in · two hops of fan-out
flowchart TD
  A["Admin server<br/>hash of channel id"]:::service
  C["Channel server<br/>owns C042 · per-channel sequence"]:::hot
  G1["Gateway · region A<br/>412 clients"]:::core
  G2["Gateway · region B<br/>36 clients"]:::core
  G3["Gateway · region C<br/>3 clients"]:::core
  U@{ icon: "sd:users", label: "Subscribed clients<br/>one websocket each" }
  A ==> C
  C ==> G1
  C ==> G2
  C --> G3
  G1 ==> U
  G2 ==> U
  G3 --> U
  class U client

Key flows#

DiagramAn @here from send to every subscribed client

%% title: An @here from send to every subscribed client
sequenceDiagram
  autonumber
  participant M as Sender
  participant W as Webapp
  participant V as Vitess
  participant A as Admin server
  participant C as Channel server
  participant G as Gateway servers
  participant O as Members
  M->>W: POST message · @here
  W->>W: auth · membership · may notify
  W->>V: insert · routed by channel id
  V-->>W: stored
  W-->>M: ok
  W->>A: message for C042
  A->>C: owner of C042 by hash
  C->>G: one copy per subscribed gateway
  G->>O: websocket push
  Note over W,O: then jobs · notify · unfurl · index

DiagramA channel server dies · ring repair and resync

%% title: A channel server dies · ring repair and resync
sequenceDiagram
  autonumber
  participant R as CHARM and Consul
  participant N as New channel server
  participant G as Gateway server
  participant C as Client
  participant W as Webapp
  alt owner healthy
    G->>C: events with the next sequence
  else owner died
    R->>N: replacement in under 20 s
    R-->>G: new ring
    G->>N: resubscribe moved channels only
    G-->>C: resync C042
    C->>W: fetch the gap from history
    W-->>C: missing messages · stored before send
  end

DiagramA job backlog waits in Kafka, not in Redis

%% title: A job backlog waits in Kafka, not in Redis
sequenceDiagram
  autonumber
  participant W as Webapp
  participant K as Kafkagate and Kafka
  participant J as JQRelay
  participant R as Redis
  participant X as Workers
  W->>K: enqueue job · leader ack
  alt workers keeping up
    J->>R: relay at full rate
    R->>X: run job
  else database slow
    J->>J: rate limit turned down
    Note over K: backlog waits on disk · 2 days retention
    Note over R: memory stays flat
  end

Data model#

DiagramCore entities

%% title: Core entities
classDiagram
  class Org {
    +orgId : string
    +workspaces : list · thousands for the largest
  }
  class Workspace {
    +workspaceId : string
    +orgId : string · Enterprise Grid parent
  }
  class Channel {
    +channelId : string · C prefix
    +ownerWorkspaceId : string · one copy
    +name : string
    +isPrivate : bool
  }
  class SharedChannel {
    +workspaceId : string
    +channelId : string
    +nameOverride : string
    +isPrivateOverride : bool
  }
  class Membership {
    +userId : string · by-user keyspace
    +channelId : string · by-channel keyspace
  }
  class Message {
    +channelId : string · shard key
    +ts : string
    +authorId : string
    +text : string
  }
  Org "1" --> "many" Workspace : parent of
  Workspace "1" --> "many" Channel : originates
  Channel "1" --> "many" SharedChannel : shared via
  Channel "1" --> "many" Membership : has
  Channel "1" --> "many" Message : stores

Messages are sharded by channel ID; membership is written twice, by user and by channel, so neither common query scatters.
A shared channel has one channels row with the owner and one shared_channels row per workspace, with its own name and privacy.

Lifecycle#

DiagramA message, from post to deletion

%% title: A message, from post to deletion
stateDiagram-v2
  [*] --> POSTED: client sends
  POSTED --> REJECTED: not a member
  POSTED --> STORED: written to the channel's shard
  STORED --> DELIVERED: channel server fan-out
  STORED --> PENDING_RESYNC: owner died mid-send
  PENDING_RESYNC --> DELIVERED: client fetches the gap
  DELIVERED --> INDEXED: index job
  INDEXED --> DELETED: deleted or retention
  DELETED --> [*]
  REJECTED --> [*]
  class INDEXED ok
  class REJECTED bad
  class PENDING_RESYNC queue

Decisions#

Decision	Chose	Over	Because
Unit of routing	Channel ID on a hash ring	Process per workspace	tenants overlap · load spreads
Shared channels	One copy · per-side rows	A copy per company	one history, one order
Fan-out	One copy per gateway	One per client from core	small core and cross-region cost
Boot	Slim boot · edge cache	Full team snapshot	cost tracks the screen
Sharding	Vitess by channel and user	Workspace shards	biggest tenant outgrew a host
Jobs	Kafka in front of Redis	Redis alone	a backlog can't stop enqueues
Search access	Channel filter in the engine	Filter after search	correct ranking and paging

When it breaks#

Failure	What users see	Contain it with
Channel server dies	under 20 s delay, some channels	ring repair · resubscribe · resync
Cache empties at peak	boots fail	throttle boots · fetch only misses · by-channel table
Job backlog	late notifications	Kafka buffer · relay rate dial
One zone half-fails	errors everywhere	siloed services · edge zone drain
Network hub saturates	Slack down	pre-scale · panic mode · monitoring off the path
Stale LB backends	failed requests at scale-down	dynamic endpoints · alert on slot use

Say it in the interview#

Route by the channel, govern by the organisation: shared channels and Grid break workspace sharding, so membership decides access everywhere.
Hash channels, not tenants: a channel server failure is a 20-second delay for a sliver of everyone, not an outage for one customer.
Store, then send: a message is in Vitess before any client sees it, so a missed live copy is a history fetch.
Two hops of fan-out: one copy per gateway server, then one per websocket.
Boot what's on screen: slim boots and an edge cache with team affinity turn the Monday herd into one core read per workspace per region.
A cache the database needs is capacity: watch scatter queries, fetch only misses, throttle boots before the loop closes.
Put side effects on a queue that never says no: Kafka in front of Redis, with the relay as the throttle.

Slack, the whole design.