250 million timelines in memory: cache IDs, not tweets

transcript

250 million home timelines. Every one kept in memory. The trick that makes it fit? Store almost nothing.

About 300,000 timelines are read every second. Each one has to load in under 200 milliseconds. Reads beat writes 50 to 1. So the read path has to be memory.

Each timeline keeps the newest 800 entries. But not the tweets. Just their IDs, about 20 bytes each. That's 16 kilobytes a user. Tiny.

Cache full tweets instead, about 1 kilobyte each? Roughly 200 terabytes. IDs only? 250 million users times 16 kilobytes. About 4 terabytes. Even with three replicas, 12 terabytes. That fits in a Redis cluster.

But here's the catch. You can't read an ID. So how do you see tweets? The cache is split by user. So your whole list comes from one shard. Then one multi-get fetches the bodies and authors. Deleted or blocked? Dropped right here. Your timeline lands in under 200 milliseconds.

200 terabytes or 4. Same timelines. Roughly fifty times smaller. Cache IDs, not tweets. Hydrate on read.

Full breakdown in the lecture.