Want it harder? Hide the model answer and work each phase in a scratchpad against its time budget, then reveal and score yourself. Or simply read on: the checkpoints below work either way.
How Do Two People Type Into the Same Sentence at the Same Moment and Both End Up With the Same Text?
It is 11:52 pm and the launch announcement goes out at midnight. Ananya is fixing the first paragraph of the press release. Kabir, in another city, is in the same paragraph rewriting the headline. Ishaan is bolding product names. Forty more people from the launch team have the document open, their coloured cursors blinking in the margins.
Ananya types "the" just as Kabir deletes the four words before it. On her screen the word appears instantly. On his, his deletion happened first. A few hundred milliseconds later both screens must show exactly the same sentence, with her word in the place she meant, his words gone, and Ishaan's bold still on the right letters. And at 12:05, when the CEO asks "what did it say at 11:45?", someone has to be able to show her.
- 100 — people with view, edit or comment access who can work in one Docs, Sheets or Slides file at the same time; past 100, "only the owner and some users with editing permissions can edit" (Google help centre, as of October 2026)
- 1.02 million — characters in one document, at most; a converted text file can be up to 50 MB (Google help centre)
- 40 — named versions per document; 15 per spreadsheet (Google help centre)
- 3 — kinds of change that every edit "boil[s] down to": inserting text, deleting text, and applying styles to a range of text (Google Drive blog, September 2010)
- 2010 — the year Google rebuilt the Docs editor around operational transformation, for "character-by-character real time collaboration" (Google Drive blog, May 2010)
For context:
Google described its collaboration protocol in a three-part series on its Drive blog in September 2010
Google hasn't published how many people use Docs itself, so this quest doesn't use a user count
Everything not in the list above is an illustrative design target, not Google's internal architecture
This is the system we'll design today.
Google Docs lets up to 100 people work in one file, in documents of up to 1.02 million characters with 40 named versions; one 100-person document takes 200 changes a second and sends ~20,000 messages. Each edit applies locally at once; one document server transforms late changes past newer revisions (operational transformation), appends each revision to the log if it still ends where expected, then acknowledges it; editors shift cursors themselves. The deep dives cover OT for text, one change in flight, cursors and comment anchors, and history by snapshot plus replay, then undo, crashes and merging three days offline by composition.
The Interview Begins#
You're interviewing at a company building a collaborative office suite. The interviewer opens a document and invites you; your cursor appears beside hers, labelled with your name. She types; the letters appear on your screen as she types them.
"This is a word processor in a browser. Everyone with the link can type anywhere, at the same time. Design the part that keeps everyone's text the same, and the history behind it."
People open a rich-text document in a browser or app and edit it together. Everyone sees everyone else's typing, character by character, and everyone's cursor and selection. The document is saved continuously, has a version history that can be browsed and restored, and can be edited offline.
Requirements
- Your own typing appears instantly, whatever the network
- Concurrent edits to the same text merge, and every screen converges on the same document
- Live cursors and selections for everyone in the document
- A complete revision history with named versions and restore
- Offline editing that merges back on reconnect
"Skip the layout engine, spelling and the AI features. I want the sync, the presence and the history."
Phase 1: Requirements Clarification (5 minutes)#
"First: what does a document look like to the sync layer? That decides how conflicts work."
Your Questions#
"Is the document a tree of objects, like a design file, or a sequence of characters?"
"A sequence of characters with formatting on ranges, plus a few embedded objects. Google wrote in 2010 that all edits 'boil down to three basic types of changes: inserting text, deleting text, and applying styles to a range of text'."
"If two people type into the same word at once, must both keystrokes survive?"
"Yes. That's the whole point. Last-writer-wins on a paragraph would throw someone's typing away."
"How many people in one document?"
"Usually one to three. The ceiling, from Google's help centre: up to 100 people with view, edit or comment access at the same time; beyond that, 'only the owner and some users with editing permissions can edit'. For a bigger audience you publish the document as a web page."
"How big can a document get, and what about history?"
"Up to 1.02 million characters. Version history shows who changed what and when, you can restore any version, and you can name up to 40 versions per document."
"Offline?"
"Yes, for days if need be. When the laptop comes back online its edits merge."
Functional Requirements#
- Editing: insert, delete and style text anywhere; tables, images and comments anchored to text; undo of your own changes
- Real time: everyone sees every change character by character, and every cursor and selection
- Convergence: all copies end up identical once everyone has every change
- History: every change kept in a revision log; browse versions, see who changed what, name up to 40 versions, restore any version
- Offline: edit without a connection; merge on reconnect
- Access: owner, editor, commenter, viewer; up to 100 people at once; revocable mid-session
Non-Functional Requirements#
Latency (design targets, same region)
- Your own keystroke on your screen: the next frame
- A keystroke on everyone else's screen: p95 < 200 ms, p99 < 500 ms
- Opening a 100-page document: p95 < 2 s to the first editable page
Durability and correctness
- An acknowledged change is never lost: it is in the revision log before the author's editor is told
- Every copy converges to the same text and formatting, whatever order changes arrive in
Scale
- 100 people in one document; documents up to 1.02 million characters
- Tens of millions of documents open at the evening peak
Phase 2: Back of the Envelope Estimation (5 minutes)#
"Google hasn't published Docs usage, so I'll state my assumptions and size for them. What matters is how many changes arrive, what one busy document costs, and how fast the history grows."
Before the numbers: one all-hands notes document
100 people are in the meeting-notes document; 40 of them are typing. Each typist's editor sends one change at most every 200 ms (it batches keystrokes while waiting for the server). How many changes a second reach the document's server? How many messages leave it, if every change goes to everyone else?
One strong answer
- In: 40 typists × 5 changes a second = 200 changes a second.
- Out: each change is acknowledged to its author and sent to the other 99: 200 × 99 ≈ 20,000 messages a second from one document.
- Transforms: a change waits ~150 ms for its round trip, so it is concurrent with about 200 × 0.15 = 30 others the server must transform it past: 6,000 transforms a second, a few milliseconds of CPU.
The CPU is trivial. What grows with the square of the room is the fan-out, which is one plausible reason a cap like 100 people exists (Google doesn't say why).
Traffic Estimation#
| Value | |
|---|---|
| People who open a document on a busy day | 200 million |
| Hours with a document open, each | 1.5 |
| Average concurrent sessions (× 1.5 h ÷ 24 h) | 12.5 million |
| Peak sessions (2×) | 25 million |
| Open documents (1.25 sessions each) | 20 million |
- Typing: 10% of sessions typing at any moment = 2.5 million typists × 5 changes a second = 12.5 million changes a second; design for 15 million
- Inbound: ~120 bytes a change on the wire = 1.5–1.8 GB/s across the fleet
- Acks and echoes: every change is acknowledged; most documents have one person in them, so echoes add only ~25%
Conclusion: the stream is large in total but split into twenty million small, independent documents. The design problem is ordering within a document, not throughput across them.
Memory and Server Estimation#
- One open document in memory: text and formatting (~50 KB typical) + the recent revision tail + session state ≈ 0.5 MB on average, skewed up by long documents
- Total: 20 million documents × 0.5 MB = 10 TB
- Servers: 64 GB hosts with ~48 GB for documents: 10,000 GB ÷ 48 GB ≈ 210; +30% headroom ≈ 270 document servers
- Per server: ~74,000 open documents, ~46,000 changes a second at the design peak
Conclusion: like any one-owner-per-document design, memory sets the server count; one busy document never comes close to filling a machine.
Storage Estimation#
- Log: an average day runs at about a third of peak: ~4 million changes a second × ~120 bytes = 480 MB/s ≈ 41 TB a day; ×3 replicas ≈ 124 TB a day
- One document's history: a report edited for a month might gather 50,000 revisions × 120 B = 6 MB of log for 60 KB of text: history is often 100× the document
- Snapshots: a full copy every 1,000 revisions means opening a document replays at most 1,000 changes
- Coarsening: after 30 days, keep the revisions that start and end each version-history group, not every keystroke
Conclusion: the log is the source of truth and the biggest thing you store; snapshots make it fast to open, coarsening makes it affordable to keep.
Key Metrics Summary#
| Metric | Value |
|---|---|
| Peak sessions / open documents | 25 million / 20 million |
| Changes at design peak | 15 million/s |
| One 100-person document | 200 changes/s in, ~20,000 messages/s out |
| Documents in memory | ~10 TB on ~270 servers |
| Revision log | ~41 TB a day before replication |
Requirements and estimation
Q1/3 40 people type in one document, each sending a change every 200 ms, and every change goes to the other 99. Roughly how many messages leave the server each second?
Answer & explanation
Answer: C
40 × 5 = 200 changes a second, × 99 recipients ≈ 20,000. Fan-out grows with the square of the room.
Q2/3 Why can't the editor just send the whole paragraph each time someone types?
Answer & explanation
Answer: B
Whole-value replacement is last-writer-wins; concurrent typing in the same paragraph would be lost. Changes must be small operations that can be merged.
Q3/3 A month-old report has 60 KB of text and 50,000 revisions. Why is its history so much bigger than the text?
Answer & explanation
Answer: C
50,000 × ~120 bytes is ~6 MB of log. Snapshots and coarsening of old history keep that in check.
Phase 3: High-Level Design (10 minutes)#
Sketch it first: Ananya's "the" and Kabir's delete
Ananya inserts "the" at position 40 of a paragraph. At the same moment Kabir deletes characters 20 to 35 of the same paragraph. Each change was made against the same version of the text. Who decides the order? What does each person's screen do while waiting, and what does each do when the other's change arrives?
One strong answer
- Nobody waits: each editor applies its own change at once and sends it with the revision number it was made against.
- One authority per document: a document server holds the text and the revision log and gives each change the next revision number. Say Kabir's arrives first: revision 812.
- Ananya's change is stale: it was made against revision 811, so the server transforms it past 812. Kabir removed 16 characters before position 40, so "the" moves to position 24. It becomes revision 813.
- Kabir's screen receives revision 813 already transformed and applies it: "the" lands at 24.
- Ananya's screen receives Kabir's delete; her editor transforms it past her own unacknowledged insert (her "the" sits after his range, so it doesn't change) and applies it. Both screens now read the same.
- Save: each revision is appended to the log before it is acknowledged.
"Two halves: operational transformation, which rewrites a change made against an older text so it still means the same thing; and a protocol, which makes sure only one place ever has to decide the order."
System Architecture#
%% title: Collaborative document editor architecture
flowchart TD
subgraph L1["Layer 1 · Clients"]
direction LR
WEB@{ icon: "sd:browser", label: "Browser editor<br/>local copy, pending changes" }
MOB@{ icon: "sd:phone", label: "Mobile apps<br/>same protocol" }
end
subgraph L2["Layer 2 · Edge"]
direction LR
GW@{ icon: "sd:gateway", label: "Session gateways<br/>WebSocket, route by document" }
API@{ icon: "sd:api", label: "HTTP API<br/>open, list, export" }
end
subgraph L3["Layer 3 · Control"]
direction LR
ACL@{ icon: "sd:lock", label: "Permissions<br/>owner, editor, commenter, viewer" }
RT@{ icon: "sd:router", label: "Document router<br/>document to owner server" }
end
subgraph L4["Layer 4 · One owner per document"]
direction LR
DS["Document server<br/>transforms, numbers revisions"]:::hot
PR@{ icon: "tech:redis", label: "Redis · presence<br/>cursors, selections, TTL" }
end
subgraph L5["Layer 5 · History"]
direction LR
LOG@{ icon: "sd:log", label: "Revision log<br/>append if last = N" }
SNAP@{ icon: "sd:object-storage", label: "Snapshots<br/>every 1,000 revisions" }
end
subgraph L6["Layer 6 · Around the document"]
direction LR
VH["Version history<br/>groups, named versions, restore"]:::service
CF@{ icon: "tech:kafka", label: "Kafka · change feed" }
IDX@{ icon: "sd:search", label: "Search, comments,<br/>notifications" }
end
WEB ==>|"changes, cursors"| GW
MOB ==> GW
GW ==> DS
GW -.->|"on join: role?"| ACL
GW -.->|"which server?"| RT
DS --> PR
DS ==>|"append, then ack"| LOG
DS -.-> SNAP
LOG --> VH
LOG --> CF --> IDX
API --> SNAP
class WEB,MOB client
class GW,API edge
class ACL,RT service
class PR cache
class LOG,SNAP store
class CF queue
class IDX serviceKEY INSIGHTS:
- One document server orders every change for its documents, so the server keeps one history and transforms against it; clients never have to agree with each other directly
- The revision log is the document. Google wrote in 2010 that "to display a document, we replay the revision log from the beginning"; snapshots only make that faster
- Presence is separate and disposable: cursors live in memory with a short expiry and are never in the log
- Everything else reads the change feed: search indexing, comment anchors and notifications follow the log instead of slowing the typing path
Data Flow: Two Edits to One Sentence#
%% title: Kabir's delete and Ananya's insert, merged by one server sequenceDiagram autonumber participant A as Ananya's editor participant S as Document server participant L as Revision log participant K as Kabir's editor Note over A,K: both at revision 811 K->>K: Delete 20-35, apply at once A->>A: Insert "the" at 40, apply at once K->>S: rev 811 · Delete(20, 16) A->>S: rev 811 · Insert(40, "the") S->>L: Append 812 if last is 811 L-->>S: ok S-->>K: ack 812 S-)A: 812 · Delete(20, 16) A->>A: Transform past pending insert, apply S->>S: Transform Insert(40) past 812 · now Insert(24) S->>L: Append 813 if last is 812 L-->>S: ok S-->>A: ack 813 S-)K: 813 · Insert(24, "the") K->>K: Apply, both screens match
MEANWHILE, IN PARALLEL:
- Ishaan's bold on "Nimbus" is a style change over a range; when text is inserted inside the range, the range grows to cover it, so the bold still lands on the right letters
- Forty viewers' editors receive revisions 812 and 813 and shift their own cursors locally; no cursor messages are needed for that
High-level design
Q1/3 Ananya and Kabir both edit against revision 811 and Kabir's change arrives first. What happens to Ananya's change at the server?
Answer & explanation
Answer: C
The server rewrites a stale change so that it means the same thing on the newer text, then numbers it.
Q2/3 When does Ananya see "the" on her own screen?
Answer & explanation
Answer: A
Local edits apply optimistically; transformation fixes up positions later, so nobody waits for the network to type.
Q3/3 Why are cursor positions not stored in the revision log?
Answer & explanation
Answer: C
Presence lives in memory with a short expiry; the log holds only changes to the document.
Phase 4: Deep Dives (20 minutes)#
Deep Dive 1: Operational Transformation for Text (Week 5 Concepts: Conflict Resolution)#
"A change is only meaningful against the text it was made on. OT is the set of rules for rewriting it so it means the same thing on a text that has moved under it."
This builds on Week 5, Day 4 · Conflict Resolution, which introduces OT and CRDTs as two ways to merge concurrent edits.
EASY AS 123
The document reads EASY AS 123. Kabir changes 123 to ABC: his first change deletes the three characters at positions 8 to 10. Meanwhile Ananya inserts IT'S at position 0. If Ananya's editor applies Kabir's delete exactly as he sent it, what does her screen show? What should it show, and what rule gets it there?
One strong answer
- Naively: her text is
IT'S EASY AS 123(16 characters). Deleting positions 8 to 10 removesY A, givingIT'S EASS 123. Wrong characters. - Correctly: she must shift Kabir's delete right by the 5 characters she inserted before it, to positions 13 to 15, removing
123. - The rule: an insert before a position shifts it right by the insert's length; a delete before it shifts it left. That shifting is operational transformation.
- Both ways round: Kabir's editor transforms Ananya's insert past his delete too; her insert is before his range, so it is unchanged. Both reach
IT'S EASY ASand then hisABClands in the same place on both.
This is the example Google used in its 2010 post, with John and Luiz: "He deleted the wrong characters!"
The rules. For two changes a and b made against the same text, transform(a, b) returns a': a rewritten to apply after b. Correctness is one property, called TP1 in the OT literature:
applying
bthentransform(a, b)must give the same text as applyingathentransform(b, a).
With a single server ordering every change (Google's and Wave's design), TP1 is the only property you need. Peer-to-peer OT, with no server, needs a second property (TP2) that many published algorithms turned out to violate; that is the main reason server-ordered OT won in practice.
Insert vs insert
- Earlier position stays; later one shifts right by the other's length
- Same position: a fixed tie-break (lower site ID first), used identically everywhere
- Without the tie-break, two screens can order "ab" and "ba" differently and diverge
Insert vs delete
- An insert after the deleted range shifts left; before it, stays
- An insert inside a deleted range survives at the cut: typed text is never silently lost
- The delete splits in two around the inserted text
Delete vs delete, and styles
- Overlapping deletes remove only what the other didn't already remove
- Google's example: bold over 10–20 transformed past an insert of 3 at 15 becomes bold over 10–23
- Different styles on the same range don't conflict: text can be both red and italic
# ot/text.py
"""
Operational transformation for plain text (illustrative). A change is a list
of primitive operations. transform(a, b) rewrites `a`, made against the same
text as `b`, so that it applies after `b`. Both sides of the protocol use the
same function and the same tie-break, so every copy converges (TP1).
"""
from dataclasses import dataclass
from typing import List, Tuple, Union
@dataclass(frozen=True)
class Ins:
pos: int
text: str
site: int # tie-break for inserts at the same position
@dataclass(frozen=True)
class Del:
pos: int
n: int
Op = Union[Ins, Del]
def T(a: Op, b: Op) -> List[Op]:
"""Transform one primitive op past another. May split a delete in two."""
if isinstance(a, Ins) and isinstance(b, Ins):
if a.pos < b.pos or (a.pos == b.pos and a.site < b.site):
return [a]
return [Ins(a.pos + len(b.text), a.text, a.site)]
if isinstance(a, Ins) and isinstance(b, Del):
if a.pos <= b.pos:
return [a]
if a.pos >= b.pos + b.n:
return [Ins(a.pos - b.n, a.text, a.site)]
return [Ins(b.pos, a.text, a.site)] # typed inside a deleted range: keep it
if isinstance(a, Del) and isinstance(b, Ins):
if b.pos <= a.pos:
return [Del(a.pos + len(b.text), a.n)]
if b.pos >= a.pos + a.n:
return [a]
k = b.pos - a.pos # the insert splits our range
return [Del(a.pos, k), Del(a.pos + len(b.text), a.n - k)]
# delete vs delete
if a.pos + a.n <= b.pos:
return [a]
if b.pos + b.n <= a.pos:
return [Del(a.pos - b.n, a.n)]
overlap = min(a.pos + a.n, b.pos + b.n) - max(a.pos, b.pos)
rest = a.n - overlap # only what b didn't already delete
return [Del(min(a.pos, b.pos), rest)] if rest else []
def transform(A: List[Op], B: List[Op]) -> Tuple[List[Op], List[Op]]:
"""Transform two changes made against the same text. Returns (A', B'):
A' applies after B, and B' applies after A."""
if not A or not B:
return A, B
if len(A) == 1 and len(B) == 1:
return T(A[0], B[0]), T(B[0], A[0])
if len(A) > 1:
a1, b1 = transform(A[:1], B)
a2, b2 = transform(A[1:], b1)
return a1 + a2, b2
a1, b1 = transform(A, B[:1])
a2, b2 = transform(a1, B[1:])
return a2, b1 + b2
def apply(text: str, ops: List[Op]) -> str:
for op in ops:
if isinstance(op, Ins):
text = text[:op.pos] + op.text + text[op.pos:]
else:
text = text[:op.pos] + text[op.pos + op.n:]
return text
# TP1 on the post's example: both orders give the same text
s = "EASY AS 123"
kabir, ananya = [Del(8, 3)], [Ins(0, "IT'S ", site=1)]
k2, a2 = transform(kabir, ananya)
assert apply(apply(s, ananya), k2) == apply(apply(s, kabir), a2) == "IT'S EASY AS "What OT doesn't decide for you:
- Intent at the edges. If Kabir deletes a sentence while Ananya types inside it, her words survive at the cut, out of context. That is a product choice (never lose typing), not a bug.
- Rich structure. Tables, lists and embedded objects need more operation types, and every new type multiplies the pairs to transform. Each pair needs a test that checks TP1 on random inputs.
Deep Dive 2: The Client–Server Protocol (Week 5 Concepts: Consistency Models; Week 2: Idempotency)#
"OT tells you how to transform two changes. The protocol makes sure you only ever transform against one history: the server's."
Five keystrokes on a slow network
Ananya's round trip to the server is 300 ms. She types "Hello" in 250 ms. Her editor follows Google's rule: never more than one change in flight. What does she send, when, and what does her editor hold while it waits? While she waits, Kabir's revision 812 arrives. What must her editor do with it?
One strong answer
- First keystroke "H" goes out at once as a change against revision 811. It is now sent, unacknowledged.
- "ello" arrives while "H" is in flight, so it collects in a pending buffer, composed into one change.
- When the ack for "H" arrives (as revision 813, say), the pending "ello" goes out against 813. Five keystrokes, two messages.
- Kabir's 812 arrives in between: her editor transforms 812 past her sent "H" and then past her pending "ello" before applying it to her screen, and transforms her sent and pending changes past 812 so they are still correct relative to the server's text.
- Why one in flight: every change she sends is based on a revision the server has, so the server only transforms against its own log. It never has to reason about her private history. Google Wave's whitepaper calls the result a "single state space" on the server.
The Solution:
%% title: The client's three states stateDiagram-v2 [*] --> SYNCED SYNCED --> AWAITING: local change sent AWAITING --> AWAITING_WITH_BUFFER: more local changes, held AWAITING_WITH_BUFFER --> AWAITING_WITH_BUFFER: more local changes, composed AWAITING --> SYNCED: ack AWAITING_WITH_BUFFER --> AWAITING: ack, buffer sent SYNCED: SYNCED · nothing unacknowledged AWAITING: AWAITING · one change in flight AWAITING_WITH_BUFFER: AWAITING + BUFFER · one in flight, the rest held class SYNCED ok class AWAITING core class AWAITING_WITH_BUFFER queue
- A remote revision can arrive in any state; it is transformed past whatever the client has in flight and in the buffer, and they are transformed past it.
- Acks carry the revision number, so the client always knows the newest revision it is based on.
- Resends are idempotent: every change carries
(client_id, seq); after a reconnect the client resends its in-flight change and the server, which recorded the lastseqit applied per client, acknowledges a duplicate without applying it again (Week 2's idempotency keys).
# collab/client.py
"""
The client half of the protocol (illustrative), following the shape Google
described in 2010: the last revision received, one change sent but not yet
acknowledged, and pending changes not yet sent. Remote revisions are
transformed past our unacknowledged work before they touch the screen.
"""
from ot.text import apply, transform
class Client:
def __init__(self, text: str, rev: int, ws, client_id: str):
self.text, self.rev, self.ws, self.client_id = text, rev, ws, client_id
self.sent = [] # at most one change in flight
self.pending = [] # local ops not yet sent, composed into one change
self.seq = 0
def local(self, ops) -> None:
self.text = apply(self.text, ops) # on screen this frame
self.pending += ops
self._flush()
def _flush(self) -> None:
if self.sent or not self.pending:
return
self.sent, self.pending = self.pending, []
self.seq += 1
self.ws.send({"rev": self.rev, "ops": self.sent, "client": self.client_id, "seq": self.seq})
def on_ack(self, rev: int) -> None:
self.rev, self.sent = rev, []
self._flush()
def on_remote(self, rev: int, ops) -> None:
ops, self.sent = transform(ops, self.sent) # past what's in flight
ops, self.pending = transform(ops, self.pending) # and past what's held
self.text = apply(self.text, ops)
self.rev = rev
def on_reconnect(self, ws) -> None:
self.ws = ws
ws.send({"hello": True, "rev": self.rev}) # send me everything after rev
if self.sent: # same seq: a duplicate is harmless
ws.send({"rev": self.rev, "ops": self.sent, "client": self.client_id, "seq": self.seq})# collab/server.py
"""
The document server (illustrative). One owner per document. A change made
against revision r is transformed past every revision committed after r,
appended to the log only if the log still ends where we think it does,
and then acknowledged and broadcast.
"""
from ot.text import apply, transform
class StaleOwner(Exception):
pass
class DocServer:
def __init__(self, doc_id: str, text: str, rev: int, log_store):
self.doc_id, self.text, self.rev, self.store = doc_id, text, rev, log_store
self.recent = {} # rev -> ops, a tail kept in memory for transforms
self.last_seq = {} # client_id -> highest seq applied
def receive(self, msg: dict, session, can_edit: bool) -> None:
if not can_edit:
session.send({"reject": "VIEW_ONLY", "seq": msg["seq"]})
return
if msg["seq"] <= self.last_seq.get(msg["client"], 0):
session.send({"ack": self.rev_of(msg), "seq": msg["seq"]}) # duplicate after reconnect
return
ops = msg["ops"]
for r in range(msg["rev"] + 1, self.rev + 1): # everything the client hadn't seen
ops, _ = transform(ops, self.history(r))
new_rev = self.rev + 1
if not self.store.append_if_last(self.doc_id, new_rev, ops, msg["client"], msg["seq"],
expect_last=self.rev):
raise StaleOwner(self.doc_id) # someone else owns this document now
self.text, self.rev = apply(self.text, ops), new_rev
self.recent[new_rev] = ops
self.last_seq[msg["client"]] = msg["seq"]
session.send({"ack": new_rev, "seq": msg["seq"]})
session.broadcast_others({"rev": new_rev, "ops": ops})
def history(self, r: int):
return self.recent.get(r) or self.store.read(self.doc_id, r) # an old base: read from the log
def rev_of(self, msg: dict) -> int:
return self.store.rev_of(self.doc_id, msg["client"], msg["seq"])Durable before acknowledged, and fenced by the revision number#
- Append, then ack. The server appends each revision to the log before acknowledging it; the log groups appends every few milliseconds, so the cost is a few milliseconds, not a round trip per keystroke. A crash never loses an acknowledged change, and an unacknowledged one is still in its author's editor.
- The revision number is the fence. The append is conditional: "write revision N+1 only if the last revision is N". If a stalled old owner wakes up after the document moved, its next append fails and it stops serving. (The Figma quest fences with a lock UUID; here the log's own sequence does the job.)
- One owner per document, routed by document ID, exactly as in the Figma quest: the router maps document to server, gateways cache the answer, and a server that doesn't own a document says "moved".
OT and the protocol
Q1/4 The document is "EASY AS 123". Kabir deletes positions 8 to 10 while Ananya inserts "IT'S " at 0. What does Ananya's editor do with Kabir's delete?
Answer & explanation
Answer: C
Her insert before his range moves it right by its length; then "123" is removed on both screens.
Q2/4 Two people insert different text at the same position at the same moment. What prevents their screens from disagreeing on the order?
Answer & explanation
Answer: B
TP1 needs a deterministic rule for equal positions; both sides of the protocol must use the same one.
Q3/4 Why does the client send at most one change at a time?
Answer & explanation
Answer: D
The server transforms only against its own log and never has to track each client's private branch.
Q4/4 A stalled old owner of a document wakes up and appends revision 4,021. The log already ends at 4,025. What happens?
Answer & explanation
Answer: B
"Append N+1 only if the last is N" makes the revision number itself a fencing token.
Deep Dive 3: Cursors, Selections and Comments (Week 3 Concepts: Backpressure; Week 4: Caching)#
"In a text editor, presence has a twist: a cursor is a position in the text, and positions move whenever anyone types."
Ishaan's cursor
Ishaan's cursor sits at position 120, after "Nimbus". Ananya inserts 30 characters at position 50. Must Ishaan's editor send a new cursor position to everyone? What about Ananya's editor showing Ishaan's cursor? And what is Ishaan's comment on the word "Nimbus" anchored to?
One strong answer
- No message needed for typing elsewhere: every editor applies Ananya's insert and shifts every cursor it knows about by the same rule as any position: 120 becomes 150 everywhere, locally.
- Messages only for real moves: clicks, arrow keys, selections, and a cursor's own owner typing. A typist's caret moves with each keystroke, but the change itself already tells everyone where it went, so a cursor message can ride along with the change.
- Cursors are positions plus a revision: "position 120 at revision 811". An editor receiving it transforms it past any revisions it has applied since 811, exactly like a change.
- Comments are anchored ranges, start and end positions transformed through every revision like cursors. If someone deletes all of "Nimbus", the range collapses and the comment shows as on deleted text rather than vanishing.
The Challenge:
- Sending everyone a cursor update for every keystroke anyone makes: 100 people × every change
- Storing cursor positions as plain numbers without a revision: they point at the wrong character after the next edit
- Keeping cursors in the revision log, so history fills with mouse clicks
- A slow phone's backlog of cursor frames delaying its document changes
The Solution:
- Presence state per session:
{session, user, colour, anchor, head, rev}held in memory on the document server and mirrored to a presence store with a 30-second expiry, so a tab that dies disappears on its own. - Coalesce: keep only the latest position per session and send one presence frame per viewer at most every 100 ms; drop frames for sockets that are behind. The next frame replaces the missed one.
- Changes before presence: on a congested connection, changes and acks go first; presence frames are skipped. Edits must never queue behind cursors.
- Viewers are cheap: viewers and commenters don't send changes; above a few dozen people, show the avatars of the rest instead of all their cursors.
# collab/presence.py
"""
Cursor and selection positions as (anchor, head, rev). Every editor transforms
positions it knows about through each revision it applies, so typing anywhere
moves everyone's cursors without any presence messages.
"""
from ot.text import Del, Ins
def shift(pos: int, op, sticky_left: bool = False) -> int:
if isinstance(op, Ins):
if op.pos < pos or (op.pos == pos and not sticky_left):
return pos + len(op.text)
return pos
if isinstance(op, Del):
if op.pos + op.n <= pos:
return pos - op.n
if op.pos < pos:
return op.pos # inside the deleted range: collapse to the cut
return pos
def transform_cursor(cur: dict, ops) -> dict:
for op in ops:
cur = {**cur, "anchor": shift(cur["anchor"], op), "head": shift(cur["head"], op)}
return cur
def transform_comment(anchor: dict, ops) -> dict:
"""A comment's range: the start sticks left, so text typed at its start stays outside it."""
for op in ops:
anchor = {**anchor,
"start": shift(anchor["start"], op, sticky_left=True),
"end": shift(anchor["end"], op)}
anchor["orphaned"] = anchor["start"] == anchor["end"]
return anchorSuggestions use the same machinery: in suggesting mode a change is an ordinary insert or delete with a "suggested by" attribute on the range. Accepting it removes the attribute; rejecting it is a new change that deletes the suggested text. History stays a list of plain changes.
Deep Dive 4: Revision History and Restore (Week 3 Concepts: Audit Log; Week 1: Replication)#
"The log is the truth. Version history is a view over it, and restoring an old version is just one more change."
This builds on Week 3, Day 5 · Audit Log System: an append-only record you can replay.
What did it say at 11:45?
At 12:05 the CEO asks for the press release as it was at 11:45, and then for it to be restored. The document has 48,000 revisions; 11:45 was revision 47,310. How do you show it fast, and how do you restore it without breaking the 40 people still editing?
One strong answer
- Show: load the nearest snapshot at or before 47,310 (snapshots every 1,000 revisions: 47,000) and replay 310 revisions. Milliseconds, not a replay of 47,310 changes.
- Restore without rewriting history: compute the change that turns the current text (48,000) into the 11:45 text, a diff, and submit it as revision 48,001 through the normal path. Everyone's editor receives it like any other change; their pending work is transformed past it.
- Nothing is lost: revisions 47,311 to 48,000 are still in the log, so the restore itself can be undone by restoring 48,000.
- The version-history panel groups revisions into versions by author and time gaps, so a user scrolls through dozens of versions rather than 48,000 keystroke batches. Named versions (up to 40 per document) are labels on revision numbers that are never coarsened away.
%% title: Opening any revision: snapshot plus replay
flowchart TD
ASK(["Show revision 47,310"]):::client
SN@{ icon: "sd:object-storage", label: "Snapshot 47,000<br/>full text and styles" }
LG@{ icon: "sd:log", label: "Revisions 47,001 to 47,310<br/>from the log" }
RP["Replay 310 changes"]:::core
OUT(["The document at 11:45"]):::ok
RES["Restore = diff to current,<br/>submitted as revision 48,001"]:::hot
ASK --> SN --> RP
LG --> RP --> OUT
OUT -.->|"if restored"| RES
class SN,LG store- Every revision is kept in full for a recent window (30 days here), so any moment can be shown exactly
- Snapshots every 1,000 revisions, and at every named version, bound replay time
- Version groups are computed from the log: consecutive revisions by the same people with no long gap become one version in the panel
- After the window, keep the revisions at group boundaries and named versions; drop the fine-grained ones in between (a design choice: Google's help centre doesn't say how long fine-grained history is kept)
- Attribution ("who wrote this sentence?") comes from the author recorded on each revision, kept with the group boundaries
Presence and history
Q1/4 Ananya types 30 characters before Ishaan's cursor. What must be sent so others see his cursor in the right place?
Answer & explanation
Answer: C
Cursors are positions; every editor transforms them through each revision it applies, locally.
Q2/4 Someone deletes the whole word a comment was anchored to. What should happen to the comment?
Answer & explanation
Answer: B
Comment anchors transform like cursors; a collapsed range is kept and flagged rather than silently lost.
Q3/4 How should "restore the 11:45 version" be applied while 40 people are editing?
Answer & explanation
Answer: D
Restoring forward keeps history intact, flows through the normal protocol, and can itself be undone.
Q4/4 Why snapshot every 1,000 revisions if the log alone can rebuild any version?
Answer & explanation
Answer: A
Opening revision 47,310 from scratch would replay 47,310 changes; with snapshots it is at most 999.
Phase 5: Scaling and Edge Cases (5 minutes)#
"Walk me through the launch night, and then the cases that bite."
A Document's Busiest Hour#
- 11:30 PM · Ten editors
- The document server holds the text and the last few thousand revisions in memory; ~40 changes a second
- 11:50 PM · The launch team piles in
- 70 people; presence frames drop to 10 a second and only nearby cursors are drawn in full
- 11:58 PM · The 101st person opens it
- The document is at its limit: the newcomer gets a read-only view that still updates live
- 12:00 AM · Published
- A named version "As published" is set on revision 48,000; a snapshot is written at that revision
- 12:05 AM · Restore to 11:45
- One new revision, 48,001; everyone's editor transforms its pending typing past it
Edge Cases#
Edge case 1: A 500 KB paste into a busy document
Problem: Someone pastes half a book into a document 40 people are editing.
Solution: It is one insert change; its transforms against other changes are cheap (positions shift by one length). The broadcast is large, so send it once per session and let slow clients fetch it by revision number instead of queueing behind it. Reject anything that would push the document past 1.02 million characters.
Edge case 2: Access removed while typing
Problem: An admin turns an editor into a viewer mid-sentence.
Solution: The permission service tells the document server, which rejects that session's next change as view-only. The editor drops its in-flight and pending changes, after offering to copy them, and reloads read-only.
Edge case 3: Undo after others have typed
Problem: Kabir undoes his headline rewrite after Ananya edited the same sentence.
Solution: Undo is the inverse of Kabir's own change, transformed past every later revision, and submitted as a new change. It removes only what he wrote and never undoes Ananya's typing.
Edge case 4: A document server dies
Problem: The server owning 70,000 documents crashes.
Solution: Each document moves to a new owner, which loads the latest snapshot and replays the log tail. Clients reconnect with their last revision, get the revisions they missed, and resend their in-flight change, which the new owner deduplicates by (client, seq).
Scaling and edge cases
Q1/3 The 101st person opens a document. What should they get?
Answer & explanation
Answer: C
Beyond 100 people only the owner and some editors can edit; publishing the document as a web page serves a bigger audience.
Q2/3 Kabir undoes his change after Ananya edited the same sentence. What does undo submit?
Answer & explanation
Answer: C
Undo removes only his own contribution, applied as a new change through the normal path.
Q3/3 A document server crashes. Why is no acknowledged change lost?
Answer & explanation
Answer: C
Append, then ack. Anything not yet acknowledged is still in its author's editor and is resent.
Phase 6: Monitoring and Operations#
"Three questions for the dashboard: does typing reach people fast, does every copy converge, and is the history safe?"
Monitoring Dashboard#
Sync
| Metric | Now |
|---|---|
| Keystroke to other screens, p50 / p95 / p99 | 61 / 154 / 390 ms |
| Changes accepted, fleet | 11.2 million/s |
| Mean transforms per change | 0.4 (max 31, one 100-person document) |
| Reconnects, last minute | 48,000 |
Correctness
- Convergence checks: clients send a hash of their text every 500 revisions; mismatches last hour: 3 (all fixed by reload, reported to the OT team)
- Duplicate changes dropped after reconnect: 0.6%
History
- Log append p99: 9 ms; conditional-append failures, last hour: 4 (all during document moves)
- Oldest snapshot lag: 1,840 revisions on one document (alert at 5,000)
Servers
- Busiest server: 81,000 open documents, 77% of memory; largest document 0.98 million characters
SLOs for the Platform#
SLO 1: No lost changes
- Target: 0 acknowledged changes lost; log append p99 under 20 ms
SLO 2: Typing reaches people fast
- Target: keystroke to other screens p95 < 200 ms in every 5-minute window, per region
SLO 3: Copies converge
- Target: convergence-hash mismatches below 1 in 10 million revisions; every one investigated
SLO 4: History opens fast
- Target: any version in the history panel shown within 2 s (p95)
Reading the dashboard
Q1/3 Clients report a hash of their text every 500 revisions. What is it catching?
Answer & explanation
Answer: A
Convergence is the property OT promises; checking it in production finds transform bugs no test found.
Q2/3 One document's newest snapshot is 1,840 revisions old and growing. What is at risk?
Answer & explanation
Answer: C
Durability comes from the log; snapshots bound replay. A stuck snapshotter makes opens and failovers slow.
Q3/3 Conditional-append failures appear only while documents are being moved. Is that a problem?
Answer & explanation
Answer: B
The revision-number fence stops a stale owner; it should fail only around ownership changes.
Final Challenge: Three Days Offline#
The interviewer scrolls to a support ticket. "This one's real enough."
"Kabir edited the 40-page quarterly report on a laptop with no connection for three days: 2,400 changes. Meanwhile 25 colleagues made 60,000 revisions to the same document. He reconnects. Merge his work without freezing the document for the people in it, and tell me what he should see."
- Kabir's base: revision 112,000; the document is now at 172,000
- Kabir's offline changes: 2,400; the document: ~120,000 characters
- One primitive transform: ~1 µs (assumption)
- 30 people are in the document right now; the server keeps fine-grained history for 30 days
How do you merge 2,400 changes past 60,000 without freezing anyone?
Estimate the naive cost, find a cheaper way, decide where the work runs, and decide what Kabir sees when his text lands somewhere it no longer makes sense.
One strong answer
- Naive: transform each of his 2,400 changes past each of the 60,000 revisions: 2,400 × 60,000 = 144 million transforms, ~144 seconds at 1 µs each. If that runs on the document's server thread, 30 people stop seeing each other's typing for over two minutes.
- Compose first: OT changes can be composed. The 60,000 revisions compose into one change, the difference between revision 112,000 and 172,000, whose size is bounded by the document (a run-length list of keep, insert and delete spans, a few thousand entries for a 120,000-character document). His 2,400 compose into one change too. Transforming one against one is linear in their sizes: milliseconds.
- Run it off the hot path: the client asks for the composed server change since 112,000 (built from the log by a background worker), transforms its buffer locally, and submits the result as one change against revision 172,000. The document server treats it like any other change; if a few revisions landed in the meantime, it transforms past those few.
- History must still exist: this only works because revisions since 112,000 are still in the log. If his base were older than the fine-grained window, fall back to a three-way merge from his base snapshot, the current text and his text, like a code merge.
- What Kabir sees: before merging, his editor saves an automatic version of the document as it was ("Before Kabir's offline edits"), so anything can be compared or restored. Where his typing landed inside text someone else deleted, OT keeps it at the cut: flag those paragraphs. A product could also submit a long-offline merge in suggesting mode, so the 30 people review it instead of finding it.
Final challenge · Quest check
Q1/4 2,400 offline changes transformed one by one past 60,000 revisions, at 1 µs per transform. How long is that?
Answer & explanation
Answer: C
2,400 × 60,000 = 144 million transforms. Composition turns it into one transform of two bounded changes.
Q2/4 Why does composing the 60,000 server revisions help?
Answer & explanation
Answer: A
Many small edits to one document compose into a single span list no bigger than the document itself.
Q3/4 Where should the expensive part of the merge run?
Answer & explanation
Answer: C
The sequencing path stays fast for the 30 people typing; the merge arrives as an ordinary change.
Q4/4 Kabir typed a paragraph inside a section a colleague deleted. What does OT do with his text?
Answer & explanation
Answer: C
Typed text is never silently dropped; the product flags it so a human can decide.
Interview Conclusion#
"Why OT and not a CRDT?"
"Because we have a server anyway, for permissions, history and search. With one server ordering changes, OT only needs one correctness property, and the document carries no per-character identifiers or tombstones. A text CRDT shines without a server, peer to peer or offline-first; here it would cost metadata for an option we don't use. The Figma quest is the opposite case: its data is properties on objects, where last-writer-wins is enough."
"Where does OT hurt?"
"Every new kind of operation multiplies the pairs you must transform and prove. Tables, lists and comments each add rules. And very long offline sessions need composition, not keystroke-by-keystroke transforms."
"The single most important decision?"
"One owner per document with a revision log, and clients that never send more than one change at a time. That gives the server a single history to transform against, makes the revision number both the ordering and the fence, and turns version history and restore into reads and writes on the same log."
Summary: Concepts Applied from 10-Week Course#
- Week 1: Data at scale
- Partitioning by document; one owner per document, routed by document ID
- Week 2: Failure-first design
- Idempotency — (client, seq) on every change makes reconnect resends harmless
- Week 3: Messaging and async processing
- Audit log — the revision log as an append-only source of truth; a change feed for search and notifications
- Backpressure — presence frames coalesced and skipped, never ahead of changes
- Week 5: Consistency and coordination
- Conflict resolution — operational transformation for text, TP1, tie-breaks
- Fencing — conditional appends on the revision number
- Week 10: Production readiness
- SLOs on keystroke latency and convergence; hash checks in production
Why Google Docs Matters#
Positions, not values
- A design file's edits set properties; a text edit's meaning depends on where everything else is. OT exists because every position moves when anyone types
One history beats many
- Google's 2010 design keeps the hard part on one server: clients send one change at a time, so the server never has to reason about anyone's private branch
History is the product
- The same log that makes the text durable gives version history, restore, attribution and undo, with no separate system to keep in sync
"Everyone types into the same sentence, nobody waits, and the log remembers every word anyone ever wrote."
Self-Assessment Checklist#
After studying this case study, you should be able to:
OT:
- Transform insert/insert, insert/delete and delete/delete pairs, and explain the tie-break
- State TP1 and check it on an example
- Explain why server-ordered OT needs only TP1
Protocol and presence:
- Describe the client's sent and pending changes and why only one change is in flight
- Make reconnect resends idempotent and fence a stale owner with conditional appends
- Keep cursors and comment anchors correct by transforming positions
History and operations:
- Open any revision with snapshots plus replay, and restore by appending a new revision
- Merge a long offline session with composition, and say where the work runs
- Compare OT here with per-property last-writer-wins and with text CRDTs
Sources#
Google Drive blog (Google Docs engineering):
- Book
Jeff Harris — What's different about the new Google Docs? (new editing surface and layout engine; built around operational transformation for "character-by-character real time collaboration"), May 2010: https://drive.googleblog.com/2010/05/whats-different-about-new-google-docs.html
- Blog
What's different about the new Google Docs: Working together, even apart (comparing versions vs. a document stored as chronological changes; paragraph locking), September 2010: https://drive.googleblog.com/2010/09/whats-different-about-new-google-docs_21.html
- Blog
John Day-Richter — What's different about the new Google Docs: Conflict resolution (three change types, revision log, the EASY AS 123 example, style transforms), September 2010: https://drive.googleblog.com/2010/09/whats-different-about-new-google-docs_22.html
- Blog
John Day-Richter — What's different about the new Google Docs: Making collaboration fast (client and server state, one pending change at a time, server transforms against committed changes), September 2010: https://drive.googleblog.com/2010/09/whats-different-about-new-google-docs.html
Google help centre (as of October 2026):
- Docs
Share & collaborate in Google Docs, Sheets and Slides (up to 100 people at once; past 100 only the owner and some editors can edit; publish to the web for more viewers): https://support.google.com/a/users/answer/9305685
- Docs
Files you can store in Google Drive (Docs: up to 1.02 million characters; 50 MB when converted): https://support.google.com/drive/answer/37603
- Docs
Check activity and file versions (version history; up to 40 named versions per document, 15 per spreadsheet): https://support.google.com/docs/answer/190843
Research and design documents:
- Read
C. A. Ellis, S. J. Gibbs — Concurrency control in groupware systems, SIGMOD 1989 (the first OT algorithm): https://doi.org/10.1145/67544.66963
- Read
David A. Nichols, Pavel Curtis, Michael Dixon, John Lamping — High-latency, low-bandwidth windowing in the Jupiter collaboration system, UIST 1995 (client-server OT): https://doi.org/10.1145/215585.215706
- Paper
David Wang, Alex Mah, Soren Lassen — Google Wave Operational Transformation, version 1.1, July 2010 (acknowledgements, a single state space on the server): https://svn.apache.org/repos/asf/incubator/wave/trunk/whitepapers/operational-transform/operational-transform.html?p=1400000
- Docs
Marc Shapiro, Nuno Preguiça, Carlos Baquero, Marek Zawirski — Conflict-free Replicated Data Types, SSS 2011: https://doi.org/10.1007/978-3-642-24550-3_29
Design note: the architecture, code, thresholds, rates, sizes and dashboards here are an illustrative design for the problem, not a description of Google's internal systems beyond what Google has published.
Further Reading#
Related systems to study:
- Read
Figma (Bonus Problem 15 in this series): multiplayer editing of objects with per-property last-writer-wins, journals and fencing
- Read
WhatsApp (Bonus Problem 2): ordering and offline delivery per conversation
- Talk
YouTube (Bonus Problem 19): rebuildable derived data from one source of truth
Foundational concepts (this course):
- Read
Week 5, Day 4 — Conflict Resolution
- Read
Week 5, Day 1 — Consistency Models
- Read
Week 5, Day 5 — Leader Election
- Read
Week 3, Day 5 — Audit Log System
- Read
Week 2, Day 2 — Idempotency in Practice
- Read
Week 3, Day 3 — Backpressure and Flow Control
"Every keystroke is a small change against a numbered revision; one server puts them in order, and the order is the document."
- 0/7stages cleared
New lessons by email
New lessons, side quests and case files as they ship, at most once a week. No spam, unsubscribe anytime.
Discussion · …
Public: everyone reading this lesson sees comments. Be kind and keep it on the material.