Cost, caching and operations: running an AI feature

Sale week at MegaMart Assist: the four token meters and why history makes cost grow with the square of the turns, provider prompt caching (prefix order, exact match, prices, and why cache reads are rate-limit headroom), exact and semantic answer caches and their false hits, routing and cascades and their break-even, batch APIs at half price, reserve-then-settle budget governors with a spend ladder, one retry layer and costed failover, OpenTelemetry spans and cost attribution, real provider incidents, runbooks, and the monthly cost model.