One expired key took down Black Friday

transcript

Black Friday. Midnight. One cache key expires. Five seconds later, the whole site is down.

The homepage key expires. 50,000 users refresh. Every single one misses. 50,000 identical queries hit the database. One second in, CPU hits 100 percent. Queries crawl. 50 milliseconds becomes 5,000. The connection pool runs dry. Five seconds in, the site is down.

A normal day: 10,000 requests a second, 99 percent hits. 100 reach the database. The database can take 1,000. Plenty of room. Now one key with half the traffic expires. 5,100 a second. That's the thundering herd. One miss becomes thousands of queries.

The fix is called request coalescing. Request A misses, and starts one fetch. B and C miss too. But they don't query. They join A. One call reaches the database. Just one. The result goes to all of them at once. No polling.

Instagram faces this with celebrity posts. Millions of views in minutes. Their trick: cache a promise for the value, not just the value. Everyone else awaits it. At most one query per key, per server.

But here's the catch. What if that one fetch hangs? Every waiter hangs with it. So always set a timeout.

A thousand misses at once? Still one database query. One miss. One query. That's your interview line.

Full breakdown in the lecture.