The best seller that was in no top 10

transcript

Your best seller can vanish from your trending board. And every server counted it.

Views are spread over 8 Kafka partitions, by user id. Each one computes its own top 10 for the hour. A final step merges the eight lists.

But here's the catch. One product is 11th on every partition. So it appears in no list at all. Eight times eleventh. Zero times on the board.

Product A is first on just one partition. It makes the board. Product B is eleventh on all eight. It never shows up. Add up B's views across all eight. It can beat A.

Fix one: key by product before you count. Each product lands in one place. Its count is complete. Fix two: send summaries, not lists. Say 1,000 counters for a top 10. Merge those.

Merged top 10 lists lose everything below each cut-off. Key by product. Or merge summaries, never lists.

Full breakdown in the lecture.