Webhook delivery: at least once, to systems you don't control

Delivering a million webhooks an hour to receivers that are slow, down or lying: why at-least-once plus idempotent receivers is the only honest guarantee, the event bus, dispatcher, partitioned queues and workers, a 39-hour retry schedule, HMAC signatures and replay protection, sizing the workers, and getting a receiver that was down for 2 hours back in 13 minutes instead of 2 hours.