200 OK, then an error after 180 tokens: why every streamed reply needs its stop reason

transcript

The server said 200 OK. Your code logged a success. 180 tokens later, it failed anyway. And your app never noticed.

Rohan asks the support bot about his refund. The provider says 200. The answer starts streaming. Then, after 180 tokens, an overload error. Mid-stream.

But here's the catch. The 200 already went out. The status code is long gone. Headers are sent. And automatic retries can't help once tokens flow.

Treat 200 as success, and half a sentence becomes the answer. Next turn, the model reads it back as history. Every later turn is poisoned. Instead, tell the browser in-band, inside the stream. An interrupted event. Store the reply as interrupted. Never as complete, however finished it looks. Back off with jitter. Then continue from the partial, or regenerate.

With streaming, errors can arrive after the 200. Plan for it. So every reply keeps the reason it stopped.

Full breakdown in the lecture.