~learn.
Code lab · World 06 · Real-Time Delivery

Deliver once, retry right.

Claim a task key before sending so a redelivered task is skipped, free the claim on failure, then classify the error and back off with full jitter, cut off at the message's expiry.

Medium~35 min3 example + 10 hidden tests+90 XPJump to the code ↓
Saving also stores the Python runtime (~13.5 MB, once for every lab) so this lab runs with no internet.
PractisesW06 · D28.2 Dedup at Every HopW06 · D29.1 Classify FirstW06 · D29.2 Backoff Bounded by Expiry

A push worker gets 200 from APNs and crashes before committing its Kafka offset. Kafka redelivers the task, and without a guard the user's phone buzzes twice (Day 2, section 8.1). The lesson's answer is at-least-once with dedup: claim the task's key before sending, record "done" after, and decide what to do with a failure from its class, not from habit (sections 8.2 to 9.2).

Implement two functions.

deliver_once(task, provider, kv, policy) returns a status string:

Situation Do Return
kv holds "done" for "sent:" + task.key nothing "duplicate-skipped"
someone else holds the claim ("sending") nothing; but if policy.at_most_once (bulk), treat it as sent "retry-later", or "duplicate-skipped" for bulk
you claimed it (kv.set(key, "sending", ttl=policy.claim_ttl_s, nx=True) returned True) provider.send(task)
...and the send succeeded kv.set(key, "done", ttl=DONE_TTL_S) "sent"
...and the send failed kv.delete(key) so a retry can claim it "failed"

next_step(task, result, now, policy, rng) decides what happens after a failed send and returns a tuple:

  1. result.permanent (APNs 410, a bad number): ("fallback", task.fallback) if the task has a fallback channel, else ("dlq", result.reason).
  2. Otherwise compute the delay: result.retry_after_s if the provider sent one, else full jitter, max(policy.min_delay_s, rng.uniform(0, policy.base_delay_s * 2 ** task.attempt)).
  3. If this was the last attempt (task.attempt + 1 >= policy.max_attempts) or the retry would land at or after task.expires_at (now + delay >= task.expires_at): a "bulk" task returns ("drop", "expired or out of attempts"); any other returns ("fallback", task.fallback) if it has one, else ("dlq", "retries exhausted").
  4. Otherwise ("retry", delay).

Rules the tests check:

  • Call provider.send at most once per deliver_once call, and never for a skipped task.
  • Draw randomness only from the injected rng (call rng.uniform once per non-permanent failure without retry_after_s).
  • The fake kv expires keys on the injected clock: a claim left behind by a crashed worker disappears after claim_ttl_s, which is how a critical task gets sent again.
Python 3.14 loads on first run (~6.5 MB once)14
Loading the editor…
not run yet · 8 s limit
press Run tests

Write your code, then run 3 example tests and 10 hidden tests. Each run is limited to 8 seconds: an infinite loop gets stopped and Python restarts, your code stays.