AI budgets rarely blow up on training. They blow up on production inference nobody was counting.
A demo runs a model a few hundred times and looks essentially free. Production runs it a few million times against messy real-world inputs — with retries, long contexts, oversized prompts, and no caching — and the bill arrives looking like a second headcount you never approved.
The gap between those two numbers is where most AI cost surprises live. It isn't a modeling problem or a vendor problem. It's a metering problem.
None of these are sophisticated. They're just rarely done before a feature ships, and painful to retrofit after.
Account-level totals tell you the bill went up. Feature-level metering tells you which product decision caused it. Without that attribution you're negotiating with a vendor when you should be fixing a prompt, and you can't tell a profitable feature from one quietly eating its own margin.
Repeated identical calls are the single most common leak I see. The same document summarized on every page load. The same classification recomputed because the result was never stored. Semantically identical prompts differing only by whitespace. Caching is unglamorous and routinely pays for itself in weeks.
A number, agreed in advance, that means stop. Pilots without an off-switch don't get cancelled — they get renewed, quietly, forever, because cancelling requires someone to declare a failure and nobody volunteers for that. Write the number down before launch, when it's still cheap to be honest.
Teams often default to the largest available model for everything, then try to optimize the bill later through discounts. Most production workloads are a mix: a small fraction genuinely needs frontier capability, and the rest is classification, extraction, and routing that a cheaper model handles at a fraction of the cost per call.
Sorting workloads by what they actually require is usually worth more than any pricing negotiation — and unlike a discount, it doesn't expire.
Good AI operations looks unremarkable: metering per feature, caching what repeats, right-sizing the model to the job, and a written rule for when to quit. There's no interesting architecture diagram in any of that.
That boredom is the return. The teams whose AI costs stay predictable aren't the ones with the cleverest models — they're the ones who instrumented before they scaled.
If AI spend is growing faster than the value it's producing, metering and workload mapping are usually where a diagnostic starts.
See how the Diagnostic works →