~learn.

Part 0732 minPreface+60 XP

Part 7: API Design for System Design.

6 stages9 code

watch it first · newThe whole lesson in a 24-minute animated lecture5 quiz checkpoints · ask the AI about any moment · captions, chapters and speed

Preface · 2 min

Preface#

Basketly is a grocery-delivery app. It's illustrative: the company, its numbers and its incidents are made up to show how API decisions fail. Its first API took a weekend to write.

Basketly's first year (illustrative)
  • Week 1 —
    Ship POST /createOrder, GET /getOrders?page=3, and total in rupees as a decimal
  • Month 4 —
    On flaky 4G the app times out on POST /createOrder and retries. The first request had worked: two orders, two charges
  • Month 7 —
    A café that orders daily has 40,000 orders. Page 1,600 of its history takes 4 seconds, and database CPU climbs every time
  • Month 8 —
    A new order arrives while a customer scrolls history. Page 2 starts with the order that ended page 1
  • Month 9 —
    The backend renames total to total_amount. App version 2.3, still on 30% of phones, shows ₹0 at checkout
  • Month 10 —
    "Download a year of invoices" runs inside the request. The gateway gives up at 60 seconds with a 504, and the user taps again

None of these were database or cache bugs. Each was an API design decision made in week one: verbs in paths, offset pagination, no idempotency key, no compatibility rules, slow work inside a request.

Part 1 introduced REST, GraphQL and gRPC in a page, and Part 4 tabled the HTTP methods and status codes. This part is about designing the contract: naming things, paging through millions of them, changing the API without breaking old clients, reporting errors, surviving retries and limits, and running work that takes minutes. In an interview you sketch it in the first ten minutes, and it shapes everything you draw next.

ExplainerThe whole lesson: API design

Basketly, an illustrative grocery app, learns in its first year that every failure was an API decision: a retried POST charged twice, page 1,600 took 4 seconds, page 2 repeated an order, a rename showed ₹0 on app 2.3, and an export hit a 504 at 60 seconds. Model resources with plural nouns and a rare :cancel verb; keep the safe and idempotent promises, prefer PATCH, and answer a stale If-Match with 412. Offset pagination walks 40,000 index entries for page 2,000 and shifts under inserts, while a keyset cursor reads about 21 at any depth behind an opaque, signed token. Additive changes need no new version; renames, type changes and new required fields break old clients. Versions go in the path, a dated header (Stripe) or a query parameter (Azure), and errors follow RFC 9457. Every non-idempotent POST takes an idempotency key, a 429 carries Retry-After, and slow work returns 202 with an operation to poll or a webhook. REST suits any caller, gRPC your own services, GraphQL your own UIs.

Part I · Stage 1 of 6 · 5 min

Part I: Resources and HTTP Semantics#

Chapter 1: What an API Contract Is#

1.1 The Simple Definition#

An API is the contract between a system and the code that calls it: which operations exist, what they take and return, how they fail, and what's promised (ordering, idempotency, limits).

The contract outlives the code behind it. You can rewrite the service or split the database and callers never notice, but you can't change what a shipped mobile app sends, and partners' servers and old phones never update on your schedule. The API is the hardest part of a system to change, so design it on purpose.

1.2 Who Calls It, and Where Each Concern Lives#

DiagramOne system, three kinds of callers, and where each API concern lives
%% title: One system, three kinds of callers, and where each API concern lives
flowchart TD
  M@{ icon: "sd:phone", label: "Mobile app<br/>old versions live for years" }
  W@{ icon: "sd:browser", label: "Web app" }
  P@{ icon: "sd:server", label: "Partner servers<br/>API keys, webhooks" }
  GW@{ icon: "sd:gateway", label: "API gateway<br/>auth · rate limits · request IDs" }
  OS["Orders service<br/>validation · idempotency · pagination"]:::core
  CS["Catalog service<br/>filters · sort · cursors"]:::core
  PG@{ icon: "tech:postgresql", label: "PostgreSQL<br/>orders, products" }
  R@{ icon: "tech:redis", label: "Redis<br/>idempotency keys, limit counters" }
  Q@{ icon: "sd:queue", label: "Job queue<br/>long-running exports" }
  WH@{ icon: "sd:webhook", label: "Webhook sender<br/>export.completed" }
  M --> GW
  W --> GW
  P --> GW
  GW --> OS
  GW --> CS
  OS --> PG
  CS --> PG
  OS --> R
  OS --> Q
  Q --> WH
  WH -.-> P
  class M,W,P client
  class GW edge
  class PG store
  class R cache
  class Q queue
  class WH external

The gateway owns cross-cutting rules: who you are, how fast you may go, a request ID on every call. The service owns meaning: what a valid order is, what "the same request again" means, what order a list comes in. The contract must suit the slowest caller to change: the mobile app.

Chapter 2: Resource Modelling and Naming#

2.1 Nouns, Not Verbs#

Basketly's first API was a list of actions (/createOrder, /getOrders, /cancelOrderNow), and every feature added another. Resource-oriented design models the things and applies a few standard operations to each. Google's API Improvement Proposals: "A typical resource-oriented API exposes a large number of resources with a small number of methods on each resource", the standard ones being "Get, List, Create, Update, Delete" (AIP-121).

Operation Method and path Returns
List a user's orders GET /v1/users/{user_id}/orders A page + next page token
Get one order GET /v1/orders/{order_id} The order
Create an order POST /v1/orders 201 + Location: /v1/orders/{id}
Change the address PATCH /v1/orders/{order_id} The updated order
Delete a saved address DELETE /v1/users/{user_id}/addresses/{address_id} 204

2.2 Naming Rules That Pay Off#

  • Plural collections, alternating with IDs. "Collection identifiers must be plural", and names "alternate between collection identifiers ... and resource IDs", as in publishers/123/books/les-miserables (AIP-122).
  • Nest only for real ownership. An address belongs to a user: /users/{id}/addresses. An order relates to a user and a store, so it lives at /orders/{id} and is listed under each parent. Deep paths make clients carry IDs they don't need.
  • Opaque string IDs. Callers mustn't parse them, so you can move from auto-increment to another scheme later. A prefix (ord_8f2k…) makes logs readable.
  • One casing everywhere, snake_case or camelCase, never both.

2.3 When It Really Is a Verb#

Cancel an order isn't just PATCH {"status": "cancelled"}: it triggers a refund, recalls a rider, notifies the store. Google's custom method keeps it on the resource: "The HTTP URI must use a : character followed by the custom verb", with POST when it has side effects (AIP-136):

http2 lines
POST /v1/orders/ord_8f2k:cancel
{"reason": "ordered_by_mistake"}

POST /v1/orders/{id}/cancel is a common equivalent. Either way, verbs stay rare, explicit and attached to a resource.

Chapter 3: Methods and Status Codes That Matter#

Part 4 has the full tables (Networking, Chapter 6). These are the decisions they leave open.

3.1 Safe and Idempotent Are Promises#

RFC 9110: a method is safe if "the client does not request, and does not expect, any state change on the origin server" (GET, HEAD, OPTIONS, TRACE), and idempotent if "the intended effect on the server of multiple identical requests with that method is the same as the effect for a single such request" (those plus PUT and DELETE; not POST). Proxies, SDKs and bots act on these promises: a GET /orders/{id}/confirm gets "confirmed" by a link-preview bot. And POST, the one common non-idempotent method, is the one that creates orders and payments (Chapter 8).

3.2 PATCH vs PUT, and Lost Updates#

PUT replaces the whole resource. Google recommends PATCH, because PUT "becomes a backwards-incompatible change to add fields to the resource" (AIP-134): when v2 adds delivery_instructions, an old client that PUTs what it read silently erases it. A field mask says which fields to touch: PATCH /v1/orders/ord_8f2k?update_mask=address.

Two managers edit one product's price from the same version, and one change is lost. Optimistic concurrency fixes it: GET returns ETag: "v17", the update sends If-Match: "v17", and if the product has changed the server answers 412 Precondition Failed (RFC 9110). The client reloads and asks again.

3.3 Choosing a Status Code#

Situation Code Notes
Created 201 With Location
Accepted, not finished 202 Link to a status monitor (Chapter 10)
Done, nothing to return 204 Typical for DELETE
Malformed request 400 Name the field
Missing or invalid credentials 401 "Who are you?"
Known caller, not allowed 403 Or 404 if admitting it exists leaks something
Conflicts with current state 409 "a conflict with the current state of the target resource" (RFC 9110)
If-Match failed 412 Client's copy is stale
Well-formed but breaks a rule 422 Many APIs use 400; pick one
Too many requests 429 With Retry-After (Chapter 9)
Your bug / overloaded 500 / 503 503 with Retry-After

Stripe's published list includes 402 for "The parameters were valid but the request failed" (a declined card) and 424 for "a failure in a dependency external to Stripe" (Stripe API docs). The client's first question about any code is "should I retry?": 4xx means fix the request first (except 408 and 429), 5xx usually means try later. A 500 for a validation error makes good clients retry something that can never succeed.

Your turn · checkpoint 1not answered

Pick the status code, five times

Choose a code for each before reading on: (a) cancel an order already out for delivery; (b) POST /v1/orders with quantity: -2; (c) user A requests user B's order by ID; (d) a price update with a stale If-Match; (e) POST /v1/invoice-exports, a three-minute job.

Think it through, or jot it down. Then compare.

One strong answer

  • (a) 409: valid request, but the order's state forbids it. Add a code like order_not_cancellable.
  • (b) 422 (or 400 by your convention), naming the field.
  • (c) 404, not 403: a 403 confirms the order ID exists.
  • (d) 412: reload and retry.
  • (e) 202 with an operation to poll. Holding the connection for three minutes is the preface's 504.

Part II · Stage 2 of 6 · 6 min

Part II: Collections#

Chapter 4: Pagination#

Paginate every collection from day one. Google: "Adding pagination to an existing RPC is a backwards-incompatible change" (AIP-158), because existing clients assume the first response is the whole list.

4.1 Offset Pagination#

The obvious design is GET /v1/users/{id}/orders?page=1600&per_page=20:

sql3 lines
SELECT * FROM orders WHERE user_id = $1
ORDER BY created_at DESC
LIMIT 20 OFFSET 31980;

It's easy to build, and clients can jump to page 7. It also has two flaws that grow with the data.

Your turn · checkpoint 2not answered

Before the numbers: how much work is page 2,000?

The café has 40,000 orders, 20 per page, served with LIMIT 20 OFFSET n and an index on (user_id, created_at). How many index entries does the database walk for page 2,000? For all 2,000 pages, if the owner scrolls through everything? And what if a new order arrives while they're on page 1?

Think it through, or jot it down. Then compare.

One strong answer

  • Page 2,000: OFFSET 39,980, so ~40,000 entries walked to keep 20. PostgreSQL's docs: "The rows skipped by an OFFSET clause still have to be computed inside the server".
  • Every page: 20 × (1 + 2 + … + 2,000) ≈ 40 million entries to show 40,000 orders. Cost grows with the square of depth.
  • The new order pushes every row down one, so page 2 repeats the order that ended page 1. A delete does the opposite and one order is never shown.

4.2 Why Offset Breaks at Scale, Worked Through#

Page (20 per page) OFFSET Index entries walked Useful rows
1 0 20 20
100 1,980 2,000 20
1,000 19,980 20,000 20
2,000 39,980 40,000 20

Cost page n costs n times page 1. Most users stop by page 3, so averages look fine, but one café owner, export script or crawler on deep pages can outspend thousands of normal users. Slack hit this: "As the offset increases ... the database still has to read up to offset + count rows from disk" (Slack Engineering, 2017).

Correctness an offset counts positions, and positions move when rows are inserted or deleted above them:

DiagramOffset pagination on a list that changes between requests
%% title: Offset pagination on a list that changes between requests
sequenceDiagram
  autonumber
  participant A as App
  participant API as Orders API
  participant DB as Orders table
  A->>API: GET /orders?page=1 (offset 0)
  API->>DB: ORDER BY created_at DESC LIMIT 20 OFFSET 0
  DB-->>API: O1 ... O20
  API-->>A: page 1 ends with O20
  Note over DB: A new order N arrives at the top
  A->>API: GET /orders?page=2 (offset 20)
  API->>DB: LIMIT 20 OFFSET 20
  DB-->>API: positions 21 to 40, now O20 ... O39
  API-->>A: page 2 starts with O20 again
  Note over A: O20 shown twice. A delete would skip one instead

Slack: "the page window becomes unreliable, potentially skipping or returning duplicate results" (Slack Engineering, 2017).

4.3 Cursor (Keyset) Pagination#

Instead of "skip 39,980 rows", say where you stopped, and let the index jump there. This is the seek or keyset method (Use The Index, Luke: "No Offset"):

sql5 lines
SELECT * FROM orders
WHERE user_id = $1
  AND (created_at, id) < ($2, $3)     -- the last row the client saw
ORDER BY created_at DESC, id DESC
LIMIT 21;                             -- page_size + 1 tells us if there's more

Every page walks about 21 entries, page 1 or page 2,000, and a new order lands above the cursor where it can't shift the next page. The id tie-breaker is required: without it, two orders with the same created_at can land on both pages or neither. The API hides the SQL behind an opaque token:

python_fingerprint · 36 lines
import base64, hashlib, hmac, json

SECRET = b"rotate-me"          # signs tokens so clients can't edit them
MAX_PAGE_SIZE = 100

def _fingerprint(user_id, filters):            # a token is only valid for the query that made it
    return hashlib.sha256(json.dumps([user_id, filters], sort_keys=True).encode()).hexdigest()[:16]

def encode_token(last_row, fp):
    raw = json.dumps({"k": [last_row["created_at"], last_row["id"]], "q": fp}).encode()
    sig = hmac.new(SECRET, raw, "sha256").hexdigest()[:16]
    return base64.urlsafe_b64encode(raw).decode() + "." + sig

def decode_token(token, fp):
    body, sig = token.rsplit(".", 1)
    raw = base64.urlsafe_b64decode(body)
    if not hmac.compare_digest(sig, hmac.new(SECRET, raw, "sha256").hexdigest()[:16]):
        raise ValueError("invalid page_token")                          # → 400
    data = json.loads(raw)
    if data["q"] != fp:
        raise ValueError("page_token was issued for a different query")  # → 400
    return data["k"]

def list_orders(db, user_id, filters, page_size=20, page_token=None):
    page_size = max(1, min(page_size or 20, MAX_PAGE_SIZE))   # coerce down, don't reject
    fp = _fingerprint(user_id, filters)
    sql, args = "SELECT id, created_at, total_paise, status FROM orders WHERE user_id = %s", [user_id]
    if filters.get("status"):
        sql += " AND status = %s"; args.append(filters["status"])
    if page_token:
        sql += " AND (created_at, id) < (%s, %s)"; args += decode_token(page_token, fp)
    sql += " ORDER BY created_at DESC, id DESC LIMIT %s"
    rows = db.query(sql, args + [page_size + 1])
    more = len(rows) > page_size
    rows = rows[:page_size]
    return {"orders": rows, "next_page_token": encode_token(rows[-1], fp) if more else ""}

What the published guidelines insist on:

  • Opaque tokens. "Page tokens ... must be opaque (but URL-safe) strings, and must not be user-parseable" (AIP-158). Slack Base64-encodes "information on how to retrieve the next set of results" (Slack Engineering, 2017). Opacity lets you change the scheme per endpoint.
  • Empty token = end. That "is the only way to communicate 'end-of-collection'" (AIP-158). A short page isn't the end: filters can return fewer rows.
  • Cap, don't reject. Google coerces a large page_size "down to the maximum permitted page size" (AIP-158). Stripe's limit is "between 1 and 100", default 10. GitHub's per_page maximum is "100" for most endpoints.
  • Same query across pages. "All other parameters ... must match the call that provided the page token" (AIP-158). The fingerprint enforces it.

The shapes vary. Stripe uses object IDs as cursors (starting_after=obj_foo) with has_more. Microsoft's Azure guidelines return {"value": [...], "nextLink": "{opaqueUrl}"} and omit nextLink on the last page. GitHub puts URLs in a Link header (rel="next", rel="last") and says each endpoint uses "the page, before/after, or since query parameters": page numbers where they're cheap, cursors where they aren't.

4.4 The Trade-off#

Offset vs cursor pagination

Offset / page number

  • Cost: grows with depth
  • Under concurrent writes: duplicates and gaps
  • Jump to page 50, "page 7 of 2,000": yes
  • Fits: small, bounded lists; admin tables

Cursor / keyset

  • Cost: constant per page, given an index on the sort key
  • Under concurrent writes: stable
  • Jump to page 50: no. Slack: "The client can't jump to a specific page"
  • Fits: feeds, histories, infinite scroll, sync, exports, public list APIs

Default to cursors. Use page numbers only when the product needs "go to page 7" and deep pages are cheap. A total count is optional either way (AIP-158 makes total_size a "may"), and COUNT(*) on a big table is expensive in its own right.

Chapter 5: Filtering, Sorting and Sizing a List#

5.1 Filtering and Sorting#

The simple style is one parameter per field: ?category=dairy&in_stock=true. Google uses one filter string in a small language ("a AND b", "a < 42") and returns INVALID_ARGUMENT for a malformed one (AIP-160). For sorting, order_by is "a comma separated list of fields" with a " desc" suffix (AIP-132).

Whatever the syntax, three rules keep a list fast:

  1. Every filter and sort you expose has an index behind it. An unindexed filter becomes a full scan the first time a partner calls it in a loop. Publish the allowed fields and reject others with a 400.
  2. The cursor holds the sort key plus a unique tie-breaker: (price, id) for price order, (rank, id) for popularity.
  3. A token from one sort or filter is rejected for another, as the fingerprint does.
http7 lines
GET /v1/stores/st_12/products?category=dairy&in_stock=true&order_by=rank%20desc&page_size=40

200 OK
{
  "products": [ {"id": "prd_a91", "name": "Toned milk 500 ml", "price_paise": 2800, "in_stock": true} ],
  "next_page_token": "eyJrIjogWzk3LCAicHJkX2E5MSJdLCAicSI6ICIzNDAzZDE0ZjM3NzgxYjBmIn0=.b42b9ae5c719db64"
}

Prices are integer paise, not floating-point rupees: a float can't hold ₹0.10 exactly, and changing the type later breaks clients (Chapter 6).

5.2 Sizing a List Endpoint#

WorksheetBasketly's catalog list endpoint (every number is an assumption)
Value
Daily active users (assumed) 8 million
Category pages per user per day (assumed) 12 → 96 million requests a day
Average rate 96M ÷ 86,400 s ≈ 1,100/s
Evening peak (assumed 4× average) ~4,400/s
Page 40 products × ~250 B JSON (assumed) + ~500 B envelope ≈ 10.5 KB
Peak egress, uncompressed 4,400 × 10.5 KB ≈ 46 MB/s ≈ 370 Mbps
Peak egress, gzip at ~4:1 (assumed) ~90 Mbps
Per month, compressed 96M × 10.5 KB × 30 ÷ 4 ≈ 7.5 TB (images come from the CDN)
Index entries per request, keyset page size + 1 = 41 → ~180,000/s at peak
One crawler request at page 250 with OFFSET ~10,000 entries, the work of ~244 normal requests
API servers at ~1,000 requests/s each (assumed) 5 at peak, 6–8 with room to lose a zone

Conclusion: the bytes are small, under 0.1 Gbps compressed, and a handful of servers carries the traffic. What decides the cost is query shape: keyset on an index over (store, category, rank, id) keeps every page at ~41 reads, and a capped page_size with no unindexed sorts stops one client paging to 250 at the cost of hundreds of users. The next step is caching the first page per store and category, which most traffic reads.

Part III · Stage 3 of 6 · 4 min

Part III: Contracts That Last#

Chapter 6: Versioning and Backward-Compatible Evolution#

6.1 Compatible and Breaking Changes#

Most changes need no new version if you follow the rules. Google's AIP-180 and GitHub's versioning docs list nearly the same ones:

Change Verdict
Add an endpoint, an optional request field, a response field Safe: "New components ... may be added" (AIP-180)
Add a value to a response enum Careful: "appropriate caution should be used" (AIP-180)
Remove or rename a field Breaking: "Renaming a component is semantically equivalent to 'remove and add'" (AIP-180)
Change a field's type Breaking, "even if the new type is wire-compatible" (AIP-180)
Add a required request field, or make one required Breaking (AIP-180, GitHub)
Change a default value Breaking (AIP-180)
Add new validation rules Breaking (GitHub)

The client's half of the deal: ignore unknown fields and treat unknown enum values as "other" (a tolerant reader). Put it in your SDKs and docs from day one.

Your turn · checkpoint 3not answered

What breaks if you ship these?

App 2.3 is on 30% of phones and will stay there for months. Which of these break it? (1) add delivery_slot to the order response; (2) rename total to total_amount; (3) add status: "partially_delivered"; (4) make coupon_code required on POST /v1/orders; (5) change price from rupees (45.5) to paise (4550).

Think it through, or jot it down. Then compare.

One strong answer

  • (1) Safe, if the app ignores unknown fields.
  • (2) Breaks: the preface's ₹0. Add total_amount alongside total; remove total only in a new version.
  • (3) Risky: code that covers every known status may crash or hide the order. Ship an "unknown status" fallback before you need it.
  • (4) Breaks: every order from 2.3 fails validation.
  • (5) Breaks silently: same name, so the app shows ₹4,550 for ₹45.50. A type change is worse than a rename because nothing errors.

6.2 Three Ways to Version#

Strategy Who Looks like Good at Costs
Major version in the path Google: "the first part of the URI path"; "v1, not v1.0" (AIP-185) /v1/orders → /v2/orders Visible; easy to route and cache Each break is a big migration; versions run side by side for years
Dated version in a header Stripe (Stripe-Version, 2017); GitHub (X-GitHub-Api-Version, default 2022-11-28) Stripe-Version: 2017-05-24 Many small breaks, each opted into Hidden from logs and caches; needs a transform layer
Dated version in the query Microsoft Azure: "a required query parameter named api-version" with "YYYY-MM-DD" values ?api-version=2024-05-01 Explicit on every call Every client sends it every time

Stripe's design, as it described it in 2017, is worth knowing. On an account's first request it is "automatically pinned to the most recent version available", and every later call gets that version implicitly unless a Stripe-Version header overrides it. The code builds only the newest response, then version change modules "walk back through time" to the caller's version. Stripe described keeping integrations working through "almost a hundred backwards-incompatible upgrades over the past six years" (Stripe, 2017). The details have moved on since: as of October 2026 Stripe's docs say it has released new versions monthly with no breaking changes since 2024-09-30.acacia, saves breaking changes for a major release twice a year, and its newer SDKs send the version that was current when the SDK was released.

DiagramDate-pinned versions: build the newest response, then transform it back (after Stripe, 2017)
%% title: Date-pinned versions: build the newest response, then transform it back (after Stripe, 2017)
flowchart TD
  REQ["Request<br/>account pinned to 2016-07-06"]:::service
  H["Handler builds the response<br/>in the newest shape"]:::core
  C3["Change module · 2017-05-24<br/>undo: field renamed"]:::service
  C2["Change module · 2017-02-14<br/>undo: field moved into a sub-object"]:::service
  C1["Change module · 2016-10-19<br/>undo: list became paginated"]:::service
  OUT(["Response in the<br/>2016-07-06 shape"]):::ok
  REQ --> H --> C3 --> C2 --> C1 --> OUT

The changes in the boxes are illustrative; the mechanism is Stripe's.

Publish a deprecation window too: GitHub supports the previous version "for at least 24 more months" after a new one ships. Log which clients still call old shapes, so you know whom to contact first. In an interview, "/v1 in the path, additive-only changes within it" is usually enough; what matters is knowing which changes break clients.

Chapter 7: Error Contracts#

A good error answers four questions: what kind of failure (a stable machine-readable code), what to tell a human, which input caused it, and whether to retry. The status code only roughly answers the last.

RFC 9457, "Problem Details for HTTP APIs" (it "obsoletes RFC 7807"), is the standard shape, served as application/problem+json: type ("a URI reference that identifies the problem type"), title, status, detail (this occurrence) and instance, and problem types "MAY extend" it with their own members. Basketly's version, with code, param, retryable and request_id as our extensions:

http13 lines
HTTP/1.1 422 Unprocessable Content
Content-Type: application/problem+json

{
  "type": "https://api.basketly.example/errors/slot-unavailable",
  "title": "Delivery slot unavailable",
  "status": 422,
  "detail": "The 6-7 pm slot at store st_12 filled up while you were checking out.",
  "code": "slot_unavailable",
  "param": "delivery_slot",
  "retryable": false,
  "request_id": "req_5d1c"
}

The big guidelines agree on the essentials:

  • Google returns google.rpc.Status with an ErrorInfo holding a stable reason, a domain and metadata. If a method has always returned that machine-readable ErrorInfo, the message "may change over time". If clients ever saw errors without it, they "may well have had to resort to parsing" the message, so it must stay fixed (AIP-193).
  • Microsoft Azure uses {"error": {"code", "message", "target", "details", "innererror"}}, with code repeated in the x-ms-error-code header.
  • Stripe gives a type, a code, a param "to display a message near the correct form field", a message and a doc_url (Stripe API docs).

Clients branch on the code; people read the message. Then you can reword messages freely. Also: never leak stack traces or SQL (log them under the request_id), use one shape for every error including the gateway's 429s, and report all validation failures at once rather than one per round trip.

Part IV · Stage 4 of 6 · 5 min

Part IV: Retries, Limits and Slow Work#

Chapter 8: Idempotency at the API Layer#

Week 2, Day 2 · Idempotency in Practice builds the mechanism: key store, races, TTLs. The contract says four things:

  1. Every non-idempotent POST accepts a client-generated key, one per logical operation, reused on every retry. Stripe's Idempotency-Key header takes up to 255 characters, suggests V4 UUIDs, and keys may be pruned "after they're at least 24 hours old"; GET and DELETE need none, being "idempotent by definition" (Stripe API docs).
  2. A repeat returns the original response. Google's request_id: the server "should return the response for the previously successful request, because the client most likely did not receive the previous response" (AIP-155). Microsoft's equivalent is Repeatability-Request-ID.
  3. Same key, different body, is an error. Stripe "compares incoming parameters to those of the original request and errors if they're not the same" (idempotency_error). A request that "conflicts with another request that's executing concurrently" saves no result and can be retried (Stripe API docs).
  4. The TTL and scope are documented, so clients know how long a retry stays safe.

Why clients must retry blindly: a call can fail before it arrives, "midway while the server is fulfilling the operation", or "succeed, but the connection break before the server can tell its client about it" (Stripe, 2017). The client can't tell which, so the key has to make every retry safe.

Your turn · checkpoint 4not answered

The order that timed out

On 4G, POST /v1/orders times out on the client at 5 seconds; the server commits the order at 5.2. The app retries. What happens (a) with no key, (b) with the same key, (c) with the same key while the first request is still running, (d) with the same key after the user removed an item from the cart?

Think it through, or jot it down. Then compare.

One strong answer

  • (a) Two orders, two charges: the preface's month 4.
  • (b) The server finds the key, skips the work, returns the saved 201 with the first order's ID.
  • (c) It must not run twice: answer 409 "in progress, retry shortly" (Stripe documents such conflicts as retryable), or wait and return the first result.
  • (d) Reject it: a different body under the same key is a client bug. A changed cart is a new operation with a new key.

Chapter 9: Rate Limits as Part of the Contract#

Week 1, Day 3 · Rate Limiting at Scale covers the algorithms and distributed counters. The API's job is to tell clients where they stand, so good ones slow down before they're cut off.

RFC 6585 defines 429: "the user has sent too many requests in a given amount of time", and the response "MAY include a Retry-After header". It leaves what to count, and per whom, up to you. GitHub's REST API allows "60 requests per hour" unauthenticated and "5,000 requests per hour" per user, and reports on every response:

http5 lines
x-ratelimit-limit: 5000
x-ratelimit-remaining: 4987
x-ratelimit-used: 13
x-ratelimit-reset: 1761742800
x-ratelimit-resource: core

Over the limit, GitHub returns "a 403 or 429", and when retry-after is present clients "should not retry your request until after that many seconds has elapsed". It also caps concurrency ("No more than 100 concurrent requests"). X-RateLimit-* is a convention, not a standard: the IETF's RateLimit and RateLimit-Policy fields (e.g. RateLimit: "default";r=50;t=30) were still an Internet-Draft (version 11, May 2026) at the time of writing. Whatever you use, send Retry-After with every 429.

The client's half: honour Retry-After, else back off exponentially with jitter. Stripe recommends waiting "proportionally to 2^n" plus random jitter, which "will space out requests across all clients" (Stripe, 2017).

pythonsays_retry · 25 lines
import random, time, uuid
import requests

RETRYABLE = {408, 429, 500, 502, 503, 504}

def says_retry(resp):                        # e.g. 409 "in progress": the error body's `retryable` flag
    try:
        return resp.status_code >= 400 and resp.json().get("retryable") is True
    except ValueError:
        return False

def create_order(session, body, max_attempts=5):
    key = str(uuid.uuid4())                  # one logical order → one key, reused on every retry
    for attempt in range(max_attempts):
        resp = None
        try:
            resp = session.post("https://api.basketly.example/v1/orders", json=body,
                                headers={"Idempotency-Key": key}, timeout=5)
            if resp.status_code not in RETRYABLE and not says_retry(resp):
                return resp                  # success, or an error that retrying won't fix
        except (requests.ConnectionError, requests.Timeout):
            pass                             # did it land? unknown, and the key makes that fine
        wait = resp.headers.get("Retry-After") if resp is not None else None
        time.sleep(float(wait) if wait and wait.isdigit() else random.uniform(0, 2 ** attempt))
    raise RuntimeError(f"order not confirmed; show 'checking…' and look it up with key {key}")

Chapter 10: Long-Running Operations#

10.1 202 and a Status Resource#

A three-minute export can't run inside a request: gateways, load balancers and mobile networks drop it, and a second tap starts a second export. Make the work a resource:

DiagramA long-running export: accept, poll a status resource, fetch the result
%% title: A long-running export: accept, poll a status resource, fetch the result
sequenceDiagram
  autonumber
  participant A as App
  participant API as Exports API
  participant Q as Job queue
  participant W as Worker
  A->>API: POST /v1/invoice-exports (Idempotency-Key k1)
  API->>Q: enqueue job op_42
  API-->>A: 202 Accepted · Location /v1/operations/op_42 · Retry-After 5
  W->>Q: take op_42
  A->>API: GET /v1/operations/op_42
  API-->>A: 200 status running, progress 40
  W->>API: mark op_42 succeeded, file stored
  A->>API: GET /v1/operations/op_42
  API-->>A: 200 status succeeded, result.download_url (signed, 15 min)

RFC 9110's 202 means "accepted for processing, but the processing has not been completed", and the representation "ought to describe the request's current status and point to (or embed) a status monitor". Google's slow methods "should return a google.longrunning.Operation", polled with GetOperation, whose metadata carries "progress, partial failures" and whose done flag marks the end (AIP-151). Microsoft's Azure guidelines return 202 with an Operation-Location header, a retry-after while running, and these states:

DiagramThe operation resource's lifecycle (status values as in Microsoft's guidelines)
%% title: The operation resource's lifecycle (status values as in Microsoft's guidelines)
stateDiagram-v2
  direction LR
  [*] --> NotStarted: POST accepted (202)
  NotStarted --> Running: worker picks it up
  Running --> Succeeded: result stored
  Running --> Failed: error recorded
  NotStarted --> Canceled: cancel requested
  Running --> Canceled: cancel requested
  Succeeded --> [*]
  Failed --> [*]
  Canceled --> [*]
  class Succeeded ok
  class Failed bad

Rules of thumb: the POST takes an idempotency key, so a second tap returns the same op_42; terminal states never change; failures use Chapter 7's error shape; operations expire after a documented time; big results are a link to object storage, not a 2 GB body.

10.2 Polling or Webhooks#

Polling works through any firewall but wastes calls on hour-long jobs. For servers, add a webhook: POST export.completed to a registered URL. Signatures, retries and a receiver that's down for a day are covered in Week 2, Day 4 · Webhook Delivery. Keep the status resource anyway, so a receiver that missed a webhook can catch up. Mobile apps get a push notification instead.

Your turn · checkpoint 5not answered

A twenty-minute job behind a 60-second gateway

Partner merchants want "regenerate all my product images", which takes 5–20 minutes. What does the first call return, how does the merchant's server learn it's done, and what happens if the merchant's script crashes and calls again?

Think it through, or jot it down. Then compare.

One strong answer

  • POST /v1/stores/{id}/image-jobs with an Idempotency-Key → 202, Location: /v1/operations/op_…, Retry-After.
  • Done: a signed webhook image_job.completed, with GET /v1/operations/{id} as the source of truth if it's lost.
  • The rerun: the same key returns the same operation, so no second job. A new key while one runs can get 409 "a job is already running", if only one may run per store.

Part V · Stage 5 of 6 · 4 min

Part V: Choosing the Shape#

Chapter 11: REST vs gRPC vs GraphQL#

Part 1 described each style and Part 4 compared gRPC with REST as protocols. The design question is which caller each suits.

REST + JSON gRPC + Protobuf GraphQL
Best caller Anyone: partners, browsers, curl Your own services, often polyglot Your own UIs with many screen shapes
Contract OpenAPI (optional) .proto files, generated clients A typed schema
Caching HTTP caches and CDNs on GET None at the HTTP layer Hard over POST; GET "can help facilitate HTTP caching", or persisted queries (graphql.org)
Response shape Fixed per endpoint Fixed, or a FieldMask Client picks fields
Deadlines, errors Client timeouts; HTTP codes Built-in deadlines ("terminated with a DEADLINE_EXCEEDED error"); 17 status codes Often an errors array inside a 200 response
Main risk Chatty clients, over-fetching Browsers need a proxy; harder to debug by hand N+1 resolvers; costly queries from outside

Published examples:

  • Google defines APIs once in protobuf and maps each method to REST with google.api.http annotations such as get: "/v1/{parent=publishers/*}/books" (AIP-132), so one definition serves gRPC and JSON callers.
  • Netflix: "At Netflix, we heavily use gRPC for the purpose of backend to backend communication." Because "some response fields can be expensive to compute", callers send a protobuf FieldMask naming the fields they need. The catch Netflix notes: field names now appear in requests, so renaming one breaks callers (Netflix Technology Blog, 2021).
  • GitHub added GraphQL in 2016 because "The REST API is responsible for over 60% of the requests made to our database tier", and responses "simultaneously sent too much data and didn't include data that consumers needed" (GitHub, 2016). It then had to price queries: 5,000 points an hour per user, a required first/last of 1–100 on every connection, and at most "500,000 total nodes" per call (GitHub GraphQL docs).
  • Shopify: "The REST Admin API is a legacy API as of October 1, 2024. Starting April 1, 2025, all new public apps must be built exclusively with the GraphQL Admin API" (Shopify docs).

GraphQL moves work to the server: the N+1 problem, "commonly solved by a batching technique ... like Facebook's DataLoader", and the need for "limiting operation depth and breadth, and query complexity analysis" before it faces the internet (graphql.org).

DiagramChoosing an API style for one caller
%% title: Choosing an API style for one caller
flowchart TD
  S{"Who calls it?"}
  EXT{"External developers<br/>or browsers?"}
  INT{"Your own services,<br/>high volume or streaming?"}
  UI{"Your own UIs with<br/>many different screens?"}
  REST(["REST + JSON<br/>cursors, idempotency keys, versions"]):::ok
  GRPC(["gRPC + Protobuf<br/>deadlines, FieldMask, codegen"]):::ok
  GQL(["GraphQL or a BFF<br/>with cost limits, DataLoader"]):::ok
  S --> EXT
  S --> INT
  S --> UI
  EXT -->|"yes"| REST
  INT -->|"yes"| GRPC
  INT -->|"no, simple CRUD"| REST
  UI -->|"yes, fast-changing"| GQL
  UI -->|"few screens"| REST

Most systems use several: REST at the edge, gRPC inside, and sometimes GraphQL or a backend-for-frontend (BFF) for the apps. In an interview, default to REST for the public API and name the alternative you rejected.

Your turn · checkpoint 6not answered

Three callers, three choices

Basketly has three new callers: (1) merchants' inventory systems, in whatever language each merchant uses; (2) the pricing service, called 30,000 times a second by checkout and catalog services in Go and Java; (3) a home screen of six widgets the product team changes every sprint. Pick a style for each, with one risk.

Think it through, or jot it down. Then compare.

One strong answer

  • Merchants: REST + JSON. Any language, debuggable with curl, webhooks for "stock changed". Risk: chatty syncs, so add a bulk endpoint and cursors.
  • Pricing: gRPC. Generated clients, deadlines down the call chain, small binary messages at 30k/s. Risk: not callable from browsers, which is fine here.
  • Home screen: GraphQL or a BFF owned by the app team: one round trip, exactly the fields each widget needs. Risk: N+1 and unbounded queries, so DataLoader, cost limits, persisted queries.

Chapter 12: Real-Time APIs at a Glance#

Part 4 explains the protocols; Week 6, Day 3 · Real-Time Channels runs them at scale. As API shapes:

Need Shape Example
Rare updates, seconds of delay fine Polling with ETag/If-None-Match → 304 Order status every 15 s
Server → client stream over plain HTTP SSE: GET /v1/orders/{id}/events, resume with Last-Event-ID Live tracking, notification bell
Both directions, high frequency WebSocket with your own message types, acks, resume Chat with the rider
Server → another server Webhook to a registered URL order.delivered to a merchant

Across all four: every event carries an ID and sequence so clients can resume, the stream carries notifications, not the record (the client can always GET the truth), and the stream is authorised like any other resource.

Chapter 13: Auth Basics for APIs#

  • API keys identify a calling application or account: long-lived secrets on every request, server to server. Stripe's types show the range: a publishable key that is "safe to expose" but "can't perform sensitive operations", restricted keys "with permissions you control", and secret keys with "unrestricted permissions", which Stripe now steers new uses away from. "Only publishable keys are safe to expose outside your application's back end."
  • OAuth 2.0 access tokens let "a third-party application ... obtain limited access to an HTTP service" on a user's behalf (RFC 6749). They carry scopes, expire, and travel as Authorization: Bearer …. Use them when a user grants an app access ("let this budgeting app read my orders").

The rules: no credentials in URLs (they end up in logs); least privilege per key and token; 401 for bad credentials, 403 (or 404) for no access; and rotation without downtime (Stripe keeps old and new keys working "for up to 7 days"). Week 9, Day 5 · Security Architecture goes deeper.

Part VI · Stage 6 of 6 · 6 min

Part VI: Putting It Together#

Chapter 14: Common Mistakes#

  1. Verbs in paths. /getOrdersForUserV2 makes every feature a one-off. Use resources and standard methods, with a rare :verb.
  2. Unpaginated or offset-paginated lists. Fine at 50 items, broken at 50,000, and adding pagination later breaks clients. Cursors from day one, page_size capped.
  3. Breaking changes inside a version. Add alongside, deprecate, remove only in a new version after a published window.
  4. Retry-unsafe POSTs. Without an idempotency key, every timeout on an order or payment can become a duplicate.
  5. Slow work inside a request. Anything that can take more than a few seconds becomes 202 and an operation.

Chapter 15: API Design in the Interview#

15.1 The First 10 Minutes#

Part 1's framework puts API contracts in the deep dive, and the course plan's RADCE order (Requirements → API → Data → Core components → Extended features) puts them right after requirements. Do both: a 4-minute sketch after requirements and estimates, then detail on one endpoint if asked.

A 4-minute API sketch
  1. One line per core use case: method, path, key inputs, what comes back.
  2. Mark four properties on each line: paginated? idempotent or keyed? sync or 202? who may call it?
  3. Name the push channels: SSE or WebSocket for live updates, webhooks for partners.
  4. One sentence of contract policy: /v1, additive-only, problem-details errors, 429 with Retry-After.
  5. Move on. Every box you draw next exists to serve one of these lines.
Example dialogue: "Design a food-delivery app"

You: "Before boxes, the API: it pins down what the system must do."

http6 lines
POST /v1/orders                       Idempotency-Key · 201 + Location
GET  /v1/orders/{id}                  current state, ETag
GET  /v1/users/me/orders?page_size&page_token     cursor-paginated history
POST /v1/orders/{id}:cancel           409 once picked up
GET  /v1/orders/{id}/events           SSE: status + rider location, resumable
POST {restaurant_webhook_url}         order.created, signed, retried

"Public endpoints are REST under /v1, additive-only, with problem-details errors and per-user limits returning 429 with Retry-After. Inside, orders call pricing and dispatch over gRPC with deadlines."

Interviewer: "Why SSE and not WebSockets for tracking?"

You: "Tracking is one-way. SSE is plain HTTP, reconnects by itself and resumes from Last-Event-ID. If we add chat with the rider, that's two-way, and I'd move the app to one WebSocket for both."

❯ Say this

15.2 Key Phrases#

Phrases that show API judgement
PAGINATION

"History is cursor-paginated on (created_at, id) with an opaque token, page_size capped at 100. Offset makes deep pages cost linear in depth and duplicates rows while orders arrive."

RETRIES

"POST /orders takes an idempotency key, so a mobile retry after a timeout gets the first order back, not a second one."

EVOLUTION

"Within /v1, additive only. Clients ignore unknown fields. Renames and removals wait for /v2, with a published deprecation window."

15.3 Common Follow-Up Questions#

Question Good answer
"How does a client get page 50?" "With cursors it can't jump. Page numbers only on bounded lists; otherwise filters and search narrow the list."
"Did the order go through after a timeout?" "Retry with the same key and get the original result either way, or GET by key."
"Why not GraphQL for everything?" "Public clients could send arbitrary-cost queries, and HTTP caching gets harder. Behind our own apps, with cost limits, if screens demand it."
"How do clients know they're throttled?" "429 with Retry-After, and remaining/reset headers on every response so they slow down first."

Chapter 16: Practice Problems#

Practice · problem 1

Problem 1: The Photo Feed API#

Setup: A photo app has a home feed (newest first, from followed accounts), likes, and 200 MB video uploads. Sketch the endpoints. Which are paginated, which take idempotency keys, which return 202?

Click for solution outline
  • GET /v1/users/me/feed?page_token: cursor on (created_at, post_id); posts arrive constantly, so offsets would duplicate.
  • PUT /v1/posts/{id}/likes/me and DELETE to unlike: naturally idempotent, no key needed.
  • POST /v1/uploads returns a signed upload URL (bytes go straight to object storage); POST /v1/posts with a key returns 202 and an operation while transcoding runs.

Practice · problem 2

Problem 2: A Breaking Change Without Breaking Anyone#

Setup: GET /v1/orders/{id} returns address as a string. The product needs structured addresses (line 1, city, pincode, geo), and 18-month-old app versions are still in use. Plan it.

Click for solution outline
  • Add delivery_address as an object alongside address, filled from the same data.
  • Accept both on writes; derive the string for old readers.
  • Deprecate address, measure which app versions still read it, and remove it only in /v2 or after the window. Never turn the string into an object in place.

Practice · problem 3

Problem 3: The Partner Bulk Sync#

Setup: 2,000 merchants each sync up to 500,000 products a night. Design the list and update endpoints and the limits.

Click for solution outline
  • Read: GET /v1/products?updated_since=…&page_token=…, keyset on (updated_at, id), so a crashed sync resumes from its last token.
  • Write: POST /v1/products:batchUpdate (up to 500 items, per-item results) with a key per batch, or a bulk import returning 202.
  • Limits: a per-merchant token bucket with remaining/reset headers and 429 + Retry-After, in a lane separate from shoppers' traffic.

Summary#

PART 7 KEY TAKEAWAYS
RESOURCES
  • Plural nouns, opaque IDs, shallow nesting, standard methods, a rare POST …:verb
  • PATCH with a field mask; ETag + If-Match → 412 against lost updates
COLLECTIONS
  • Paginate from day one: keyset on (sort key, unique ID), opaque token, empty token = end
  • Offset's cost grows with depth and it duplicates or skips under writes
  • Filter and sort only on indexed fields; cap page_size
CONTRACTS
  • Additive only within a version; renames, type changes, new required fields break
  • Version in the path (Google), dated header (Stripe, GitHub) or dated query (Azure)
  • Errors: stable code, human message, field, retryable (RFC 9457)
RELIABILITY AND STYLE
  • Idempotency keys on POST; 429 + Retry-After; 202 + an operation for slow work; webhooks for servers
  • REST at the edge, gRPC inside, GraphQL or a BFF for many-shaped UIs

Further Reading#

API Guidelines#

Engineering Posts#

Standards and Protocol Docs#

this lesson in shorts · 4

play them all →

One idea from this lesson in about a minute each, made for your phone.

End of Part 07 · Part 7: API Design for System Design
Finish Part 07Stages clear as you read them (0/6 so far). Skipping ahead doesn't count: marking complete unlocks after ~6 more min of reading.

Go deeper · reading trail

all reading →

Real write-ups from the teams that built these systems. Each opens on the publisher's site in a new tab.

SlackBlog post · ~9 min

Evolving API Pagination at Slack

Michael Hahn · 2017
What
Slack's account of how its Web API moved from no pagination, to offset pagination, to cursor-based pagination as workspaces grew.
Key idea
Offsets make the database read and discard every skipped row and drift under concurrent writes; an opaque Base64 cursor fixes both and lets each endpoint change its scheme.
Why here
It is the source of this part's case against offset pagination: deep pages cost more, rows are duplicated or skipped, and the client gets an opaque token instead.
taught here: Cursor pagination
StripeBlog post · ~9 min

APIs as infrastructure: future-proofing Stripe with versioning

Brandur Leach · 2017
What
Stripe explains how it ships backwards-incompatible API changes without breaking integrations that were written years earlier.
Key idea
Accounts are pinned to a dated version on first use; code builds only the newest response, and version change modules transform it back to each caller's version.
Why here
It is the dated-header strategy in this part's versioning table, and shows why knowing which changes break clients matters more than where the version lives.
taught here: Dated API versions
GitHubBlog post · ~10 min

The GitHub GraphQL API

2016
What
GitHub's announcement of its GraphQL API, with the problems in its REST API that led to it.
Key idea
The REST API drove over 60% of database-tier requests, and responses sent too much data while missing what clients needed; one typed query fetches exactly the fields wanted.
Why here
It is the GitHub example in this part's REST vs gRPC vs GraphQL chapter: GraphQL suits callers with many response shapes, at the price of server-side query costs.
taught here: REST vs GraphQL

New lessons by email

New lessons, side quests and case files as they ship, at most once a week. No spam, unsubscribe anytime.

Discussion · …

Public: everyone reading this lesson sees comments. Be kind and keep it on the material.

Sign in to ask a question or answer one. Reading stays open to everyone.
loading the discussion…