Your consumer is slow. Nothing is lost. Until day 7.

transcript

Your consumer is falling behind. Nothing is lost. Until day 7. Then it's gone. Never read.

A work queue deletes a message once it's acked. Another reader? That needs another queue. A log keeps every message for its retention. Here, 7 days. Each group remembers its own offset. Replay just moves it back.

One stored copy. Billing and search read it at full speed. Analytics slows down. Billing doesn't even notice. Consumers pull. So a slow one never pushes back on anyone. But the log only keeps 7 days.

Day 0. Analytics starts lagging. Lag grows. Still fine. The log keeps everything for 7 days. Retention deletes whole old segments. Read or not. So once lag hits 7 days, analytics loses data it never saw.

Your reader can be slow. It can't be 7 days slow. The log won't wait for you. Retention is a deadline. So alert on consumer lag. Long before it nears retention.

Full breakdown in the lecture.