Every Google search hits a slow machine. Almost every time.

transcript

Your search asks a thousand machines at once. And almost every time, one of them is slow.

Best pizza near me. The index is split across about a thousand shards. The root sends your query to every one. In parallel. Each returns its local top results. The root merges them. But the page can't finish until the slowest shard answers.

Say each shard is fast 99 percent of the time. Sounds great. The chance all 1,000 are fast? 0 point 99 to the power of 1,000. About 0 point 004 percent. Almost every query hits a slow shard. The tail is the median.

So don't wait. Shard 7's primary isn't answering. After a few milliseconds, send the same query to its replica. Whichever answers first wins. The late primary? Simply ignored.

But here's the catch. What if both miss the deadline? Return without shard 7. Its pages miss this one list. Nobody waits. A slightly incomplete result beats a slow one.

So why is a missing shard still a critical alert? Every result list is silently missing a slice of the web. Nothing errors.

A thousand shards asked. One was slow. Nobody waited for it. At this fan-out, the tail is the median.

Full breakdown in the lecture.