The circuit breaker closed. Then it knocked the service down again.

transcript

Your circuit breaker sees the service recover. It closes. One second later, the service is down again.

Normal traffic. 1,000 requests a second. The service starts failing. The circuit opens. Traffic to it drops to zero. 30 seconds later, half open. One test request. It succeeds. So the circuit closes. And 5,000 requests a second hit at once.

Where did 5,000 come from? Demand piled up while the circuit was open. Users kept retrying. Background jobs kept queueing. The moment it closes, all of it lands.

The recovering service crashes again. So the circuit opens again. Open, half open, closed, overload. Forever. Like a traffic jam clearing. Everyone accelerates, into the next jam.

The fix: gradual recovery. After closing, let a growing share through. A quarter at 15 seconds. Half at 30 seconds. The rest are shed during the ramp. All of it by 60 seconds. No flood, so no loop.

One good test request doesn't mean it can take everything. Recovery is a ramp. Not a switch.

Full breakdown in the lecture.