One export job, stuck behind another tenant's 50,000

transcript

Quarter end. The biggest tenant queues 50,000 export jobs. A small tenant queues one. When does it run?

The largest tenant has 10 million documents. The smallest have just 10,000. A thousand times smaller. A noisy neighbour waiting to happen.

With a single first-in, first-out queue, that one job waits behind all 50,000. One tenant's quarter end becomes everyone's wait.

So every tenant gets its own queue. Each has a maximum depth by plan. Past it, enqueue fails: queue full. Workers dequeue by weight. Standard 1, Professional 3, Enterprise 10.

But here's the twist. Each weight is divided by the backlog. Weight, over one plus a tenth of the queue depth. So the deeper a tenant's backlog, the smaller its share. The small job gets through.

Paying tiers still get more share. Just not all of it. A queue per tenant, weighted by plan, shrinking with backlog.

Full breakdown in the lecture.