At 90% busy, every request waits about 9 times longer

transcript

Your servers are 90 percent busy. Plenty of room left, right? Every request now waits about 9 times its own service time.

Queueing theory. At 50 percent busy, you wait about one service time. At 80 percent, about 4 times. At 90, about 9 times. At 100, the queue is infinite.

The formula: service time, times utilisation, over one minus utilisation. Near full, that bottom number heads to zero. The wait explodes.

Now map your own curve. Up to 1,000 requests a second, latency is stable. From 1,000 to 1,500, it climbs past the SLO. From 1,500 to 1,800, severe. Users clearly feel it. At 1,800, errors. The breaking point. But it isn't your capacity.

1,800 is where it breaks. Your capacity is the last load within the SLO: 1,000. Keep 30 percent headroom below that. Plan to run at about 700.

Queues don't degrade gently. They explode near full. Never run critical resources above 70 to 80 percent.

Full breakdown in the lecture.