You added two consumers. They sit there doing nothing.

transcript

Your Kafka consumers are falling behind. So you add two more. They sit there. Doing nothing. Not one message.

First, why you panicked. Producers write 10,000 a second. Your consumer handles 1,000. Ten minutes later, you're 5 point 4 million messages behind. An hour later, 32 point 4 million.

Here's the rule. Your topic has 6 partitions. You already run 6 consumers. Inside a consumer group, each partition goes to exactly one consumer. So consumers 7 and 8 get nothing. They're just standbys.

Here's the twist. Parallelism equals the partition count. Not the consumer count. To go faster, you need more partitions.

But adding partitions later remaps keys to new partitions. So per-key ordering breaks during the change. And you can't easily reduce them again. So plan it up front. 6 to 12 for a new topic. 50 to 100 or more for high volume.

Just add more consumers only works up to the partition count. Pick your partitions like they're permanent. They nearly are.

Full breakdown in the lecture.