Two models can cost less than one, until 95 percent

transcript

Calling two models can be cheaper than calling one. Until it isn't. So where's the line?

Every turn goes to the small model first. Haiku. A scorer checks the answer. Good enough? Send it. Not good enough? Escalate to Sonnet. That turn pays twice.

With caching, a conversation on Haiku costs about 0.12 cents. On Sonnet, about 2.2 cents. The small model is about 18 times cheaper.

But here's the catch. Escalated turns pay for both models. So how many escalations before the cascade loses?

All Sonnet: 2.2 cents a conversation. Cascade, with 40 percent escalating: about 1 cent. Around 55 percent less. Break-even? About 95 percent. Almost everything has to escalate.

The real price isn't money. It's latency. Escalated turns wait for two models, one after the other. A router picks upfront. No double wait. But it decides blind.

So route the obvious cases. Cascade the uncertain middle. And if escalations creep from 30 to 70 percent? Your bill moved too.

Full breakdown in the lecture.