One post, 400 million followers: push or pull?

transcript

This one post would take 7 minutes to deliver. 400 million followers. And your feed has to reach every one.

First, a normal post. Meera's cousin shares a baby photo. It becomes a small ID. Posted. No waiting. In the background, that ID drops into 420 inboxes. Meera opens the app. Her feed starts right there.

Now an account with 400 million followers posts. That's 400 million writes. At a million writes a second? About 7 minutes. For one post. Ten posts during a final? The queue runs for over an hour.

And every other post queues behind it. That baby photo too. So here's the trick. For giants, don't push at all.

Under a million followers? Push, just like before. Over a million? Skip fan-out completely. The post just sits in a cache of recent posts. Readers pull it in when they open the feed.

Meera opens the app. The feed service reads her inbox. Plus cached posts from the few giants she follows. Merge them, sort by ID. Time is baked into the ID. Rank, and send page one. A few lookups replaced 400 million writes.

50 million fans open the app in ten minutes? Still zero fan-out. Its post is one cache entry. Its photo, one pre-warmed key on the CDN. Push for most. Pull for the few.

Full breakdown in the lecture.