One key gets 10% of all reads. More shards won't save you.

transcript

One key gets 10 percent of all your cache reads. You add more shards. Nothing changes. Here's why.

Assume a million operations a second over 20 nodes. 50,000 each. A node handles roughly 100,000. Twice the headroom. Looks fine. Then one key takes 10 percent of reads. 90,000 a second, on one node.

Here's the catch. A key hashes to exactly one shard. Add a hundred shards. That key still lives on one of them.

So stop the reads before they leave the app server. Keep the hottest keys in each app server's memory. For 1 to 5 seconds. 200 app servers, each asking once a second. 90,000 reads become about 200.

That local cache is defence one. Most hot reads never leave the app. Defence two: spread a hot key's reads across its replicas. Defence three, for extreme keys: store N copies on different shards. Read a random one.

The price? A few seconds of extra staleness. Say it out loud. Memory is easy to buy. Hot keys are the real test.

Full breakdown in the lecture.