Serving stale data saved you. Then, at second 300, it stopped.

transcript

Your cache is serving stale data. And that's what's saving you. Then at second 300, it stops. And everything hits the database.

Here's the trick. An entry is fresh for 60 seconds. Then it's stale, but still usable, until 300. A request at 61 gets the stale value instantly. One background refresh starts. At 62, it's fresh again. Nobody waited. CDNs do this too.

But here's the catch. The database is having a bad day. Every refresh fails. What happens next?

At 60 seconds, it goes stale. Normal. Refresh fails. And again. You keep serving stale. So far, so good. At 300, the stale window ends. No more stale data to serve. At 301, every request hits a database that's already struggling.

Do nothing, and the herd arrives at the worst possible time. Instead, when a refresh fails, extend the stale window. Say 60 more seconds. Keep serving the last good value. Buy the database time to recover.

Users may see slightly stale data. That beats a database on fire. Refresh failing? Extend the stale window. Don't let it expire.

Full breakdown in the lecture.