Gmail design sheet · System Design Mastery

At a glance#

Number	Value	What it drives
Mailboxes	1.5B monthly users (2019)	metadata, index and limits partitioned by mailbox
Unwanted mail	~15B blocked a day · 99.9%+ stopped (2023)	refuse cheaply at the edge; filter funnel after 250
Inbound (assumed)	~60B a day · ~1.4M / s peak	design for 2M / s; 4xx deferrals as backpressure
Filter targets	< 0.1% spam in inbox · < 0.05% wanted in spam	when unsure, inbox with a warning
Bulk-sender line	5,000 a day to Gmail (Feb 2024)	SPF + DKIM + DMARC, aligned From:, one-click unsubscribe
Mail stored (assumed)	~3.75 EB · 15 GB quota per account	hot replicas ~30 days, then erasure-coded: ~5.9 EB vs ~11 EB raw
Search (assumed)	~12,000 / s peak · ~66 MB index per mailbox	one index per mailbox, no fan-out
Receiving limit (Workspace)	60 / min · 3,600 / h · 86,400 / day	sliding windows per mailbox; floods stay local

Architecture#

DiagramAccept, queue, filter, file, serve

%% title: Accept, queue, filter, file, serve
flowchart TD
  SND@{ icon: "sd:globe", label: "Sending servers<br/>the whole internet" } ==> MX@{ icon: "sd:mail", label: "MX edge<br/>TLS · IP reputation · RCPT · limits" }
  MX -.->|"SPF, DKIM, DMARC"| DNS@{ icon: "sd:dns", label: "DNS<br/>MX · SPF · DKIM keys · DMARC" }
  MX ==>|"quorum write, then 250"| Q@{ icon: "sd:queue", label: "Delivery queue<br/>3 zones · by mailbox shard" }
  Q ==> FILT["Spam and phishing filter<br/>auth · reputation · content · user"]:::hot
  ML@{ icon: "sd:ml", label: "Models<br/>trained on spam reports" } -.-> FILT
  FILT ==> MBX["Mailbox service<br/>labels · threads · quotas"]:::core
  MBX --> META@{ icon: "sd:database", label: "Metadata<br/>per user · strongly consistent" }
  MBX --> BLOB@{ icon: "sd:object-storage", label: "Blob store<br/>by hash · hot then erasure-coded" }
  MBX --> IDX@{ icon: "sd:index", label: "Search index<br/>one per mailbox" }
  BLOB -.-> BAK@{ icon: "sd:file", label: "Offline backup<br/>out of software's reach" }
  APP@{ icon: "sd:browser", label: "Web and apps" } ==> MBX
  APP --> OUT["Outbound MTA<br/>DKIM signing · retries"]:::service
  class SND external
  class MX,DNS edge
  class Q queue
  class ML service
  class META,BLOB,BAK store
  class IDX cache
  class APP client

"250 OK" is the line: before it, refuse cheaply and the sender stays responsible; after it, deliver or bounce, never lose.
Partition by mailbox: metadata, index, quotas and limits live on the user's shard; no request needs two users' data.
Bytes apart from metadata: tiny, changing labels in a consistent store; huge, immutable bodies in a tiered blob store.

DiagramOne message at the edge: refuse cheaply, accept durably

%% title: One message at the edge: refuse cheaply, accept durably
flowchart TD
  C["Connection from an IP"]:::service
  REP{"IP reputation<br/>and rate ok?"}
  T(["4xx · try later<br/>sender keeps it"]):::bad
  P(["5xx · refused"]):::bad
  RCPT{"Recipient exists<br/>and under limits?"}
  DATA["Receive data<br/>size cap · SPF · DKIM"]:::service
  DUP{"Seen this<br/>message key?"}
  W@{ icon: "sd:queue", label: "Queue write<br/>3 zones, wait for 2" }
  OK(["250 OK<br/>now our responsibility"]):::ok
  C ==> REP
  REP -->|"overloaded"| T
  REP -->|"known bad"| P
  REP ==> RCPT
  RCPT -->|"no such user"| P
  RCPT ==> DATA ==> DUP
  DUP -->|"retry"| OK
  DUP ==>|"new"| W ==> OK
  class W queue

Key flows#

DiagramRavi's 19 MB scan reaches Meera's inbox

%% title: Ravi's 19 MB scan reaches Meera's inbox
sequenceDiagram
  autonumber
  participant R as Ravi's mail server
  participant M as MX edge
  participant Q as Delivery queue
  participant F as Filter
  participant B as Mailbox service
  participant I as Search index
  R->>M: MAIL FROM, RCPT TO Meera
  M->>M: IP ok · mailbox exists · under limits
  R->>M: DATA, 26 MB with base64
  M->>M: SPF pass · DKIM pass
  M->>Q: Write to 3 zones, wait for 2
  Q-->>M: Durable
  M-->>R: 250 OK
  Q->>F: Message
  F->>F: DMARC pass · known sender · clean content
  F->>B: Deliver to Inbox
  B-->>I: Index within seconds
  Note over F,B: the fake refund takes the same path and is filed in Spam with a warning

DiagramWhen the answer is "try later"

%% title: When the answer is "try later"
sequenceDiagram
  autonumber
  participant S as Sending server
  participant M as MX edge
  participant L as Limits and quota
  participant Q as Delivery queue
  S->>M: RCPT TO Meera, 30 MB message
  M->>L: Receiving rate and storage check
  alt mailbox full
    L-->>M: Over quota
    M-->>S: 452 4.2.2 mailbox full
    Note over S: keeps it · retries for days
  else queue under pressure
    M-->>S: 451 try again later
  else all clear
    M->>Q: Quorum write
    M-->>S: 250 OK
  end
  S->>M: Retry 30 minutes later
  M-->>S: 250 OK after space was freed

DiagramMeera's firm's reminder, a bulk sender through an ESP

%% title: Meera's firm's reminder, a bulk sender through an ESP
sequenceDiagram
  autonumber
  participant E as ESP for the firm
  participant M as MX edge
  participant A as Auth check
  participant R as Reputation
  E->>M: From the firm's domain · 5,400 to Gmail today
  M->>A: SPF and DKIM for the ESP's domains
  alt not aligned or no DMARC record
    A-->>M: DMARC fail
    M-->>E: 4.7.31 or 4.7.32 · no mitigation
  else ESP signs as the firm · DMARC published
    A-->>M: DMARC pass · identity is the firm
    M->>R: Count volume · spam reports
    Note over R: spam rate kept below 0.3% · one-click unsubscribe honoured within two days
  end

Data model#

DiagramCore entities

%% title: Core entities
classDiagram
  class Mailbox {
    +mailboxId : string · shard key
    +usedBytes : cached counter
    +quotaBytes : 15 GB shared
    +receiveWindows : minute, hour, day
  }
  class Message {
    +messageId : string
    +threadId : string
    +receivedAt : time
    +flags : unread, starred
    +blobRefs : hashes
  }
  class Blob {
    +hash : sha256
    +refCount : int
    +tier : hot replicas or cold erasure-coded
  }
  class LabelBitmap {
    +label : string
    +messageIds : bitmap
  }
  class IndexSegment {
    +postings : words with positions
    +fields : from, date, size
    +deleted : bitmap
  }
  class SenderDomain {
    +domain : DMARC-aligned identity
    +reputation : score
    +dailyCount : bulk at 5,000
    +spamRate : below 0.3%
  }
  Mailbox "1" --> "many" Message : owns
  Message "many" --> "many" Blob : body, attachments
  Mailbox "1" --> "many" LabelBitmap : labels
  Mailbox "1" --> "many" IndexSegment : search
  SenderDomain "1" --> "many" Message : sent

Everything but blobs and sender reputation is keyed by mailbox: ~33,000 messages and a ~66 MB index per mailbox (assumed).
Blobs are immutable and stored once per content hash; a newsletter's logo sent to 3 million users is one blob with a reference count.

Lifecycle#

DiagramA message, from SMTP to purge

%% title: A message, from SMTP to purge
stateDiagram-v2
  [*] --> RECEIVING: SMTP DATA
  RECEIVING --> DEFERRED: 4xx · sender keeps it
  RECEIVING --> REFUSED: 5xx
  RECEIVING --> QUEUED: quorum write · 250 OK
  QUEUED --> INBOX: filter verdict
  QUEUED --> SPAM: filter verdict
  QUEUED --> BOUNCED: undeliverable after retries
  SPAM --> INBOX: Not spam
  INBOX --> TRASHED: user deletes
  TRASHED --> INBOX: restored
  TRASHED --> PURGED: 30 days · grace period
  DEFERRED --> [*]
  REFUSED --> [*]
  BOUNCED --> [*]
  PURGED --> [*]
  class INBOX ok
  class REFUSED bad
  class BOUNCED bad
  class DEFERRED queue

Decisions#

Decision	Chose	Over	Because
When to say 250	After a quorum write to 2 of 3 zones	Acknowledge, write later	250 transfers responsibility; a crash must not lose mail
Overload	421 / 451 deferrals, low reputation first	Accept and buffer	senders must retry for days; their queues are our buffer
Filtering	Cheap signals first, models on what's left	Every message through every model	~60B messages a day (assumed); models are the cost
Sender trust	Reputation per DMARC-aligned domain	Per IP only	ESP IPs are shared; an aligned domain is an identity
Storage	Hashed immutable blobs; hot replicas, then erasure-coded 1.5×	3 full copies forever	~5.9 EB vs ~11 EB raw on the estimate
Durability	Slow purges, cell-by-cell rollouts, offline backups	Replicas alone	a software bug runs on every replica, as in 2011
Search	One index per mailbox, labels as bitmaps	One global index	no query spans users; labels change too often to re-index
Limits	Sliding windows per mailbox; 452 when full	Permanent 5xx	the user can fix it before the sender gives up

When it breaks#

Failure	What users see	Contain it with
Queue writes slow at peak	mail arrives late	451 deferrals, low-reputation senders first
Storage release deletes bodies	messages that won't open	stop the rollout; freeze purges; restore from tombstones, then backups
Mailbox full	senders' mail delayed	452 4.2.2 so senders retry; credit deletes to the counter at once
Flood to one address	one mailbox stops receiving	per-mailbox receiving limit and queue partition
Trusted sender hijacked	phishing that passes DMARC	content models still run; behaviour change; fast user reports
Mailing list breaks DKIM	list mail in spam	list signs as itself or adds ARC

Say it in the interview#

"250 OK is a promise": write to two zones before saying it; before it, refuse cheaply and leave the sender responsible.
Let SMTP absorb overload: 4xx makes senders retry for days, so a spike costs delay, not mail.
Authentication isn't trust: SPF and DKIM prove who; reputation, domain age and content decide whether to believe them.
The 2024 rules give senders an identity: above 5,000 a day, SPF, DKIM, aligned DMARC, one-click unsubscribe and a spam rate under 0.3%.
Replicas don't stop bugs: slow purges, cell-by-cell rollouts and an offline copy, the lesson of Gmail's 2011 tape restore.
A billion small indexes: search is per mailbox, on the mailbox's shard, with labels as bitmaps.
Temporary when the user can fix it: 452 for a full mailbox; three receiving windows that are one rate.

Gmail, the whole design.