Google sends every query to every shard, on purpose

transcript

Your search hits every index shard. All of them. That sounds wasteful. It's the smart choice.

50 billion pages, about 500 distinct words each. That's 25 trillion postings: one per word, per page. Compressed, with three replicas, about 180 terabytes. Far too big for one machine.

Two ways to cut it. Give each shard some words, or some pages. By word sounds efficient. A query visits only its words' shards. Here's the catch. Common words make enormous, hot shards. And joining lists from different machines moves a lot of data. By page, each shard holds every word for its pages. Load spreads evenly. A lost shard drops some pages, not whole words. Each one answers locally, in parallel. The root merges.

Now growth gets simple. Two levers. Growing from 50 to 500 billion pages? Add shards. Going from 10,000 to 100,000 queries a second? Add replicas.

Why not more shards for more queries? Every query hits every shard anyway. Replicas split the query load. Shards only grow the corpus.

Shard by document. Replicate for QPS.

Full breakdown in the lecture.