Output costs 5× more, yet input is 75% of the bill: the 3,000-token prompt sent every turn

transcript

Output tokens cost five times more than input tokens. So why is three-quarters of this AI bill input?

Claude Sonnet 5.5, list price. 2 dollars per million tokens in. 10 dollars per million tokens out. Output should dominate. It doesn't.

Here's the catch. The model is stateless. So every turn re-sends the whole system prompt. 3,000 tokens of policy.

A six-turn support chat. The prompt goes six times: 18,000 tokens. History and messages add about 5,000 more. And the replies, the expensive part? Just 1,500 tokens. 78 percent of the input is the same prompt. Again and again.

Now scale it to 60,000 chats a day. Input costs 2,761 dollars. Output: just 900 dollars. Input is about 75 percent of the bill. Finance wants to know why.

So stop shaving the replies. The levers are on the input side. Budget the history. Retrieve, don't paste. Cache the fixed prompt.

Full breakdown in the lecture.