~learn.
0:00 / 1:07
W08 · Analytics pipeline

74% of the cache was used once

transcript

74 percent of this cache was used exactly once. That's one Akamai cluster. Roughly 400 million objects.

They're called one-hit wonders. A normal cache stores everything on the first request. So three in four objects cost a disk write. And nobody ever asks for them again.

The trick: cache an object only on its second request. First request? A Bloom filter just remembers it. Second request? Now it goes on disk. One-hit wonders never get past the filter.

The byte hit rate rose from 74 to 83 percent. Disk writes fell from 10,209 a second to 5,738. That's 44 percent fewer writes. From one Bloom filter.

But wait. Bloom filters have false positives. Here, who cares? A one-hit object gets cached. That's all. A little wasted disk, now and then. Nothing lost.

Use a Bloom filter where a wrong yes is cheap. Never where a wrong yes throws data away.

Full breakdown in the lecture.