If a system can return a different answer to the same input, that's a design choice. The question worth asking of every AI feature is whether anyone actually made it.
A deterministic tool gives the same output for the same input, every time. A SQL query, a tax calculation, a pricing rule, a state machine. You can test it exhaustively, reason about it, and reproduce any result later.
A non-deterministic tool may not. Language models are the obvious case — same prompt, different phrasing, different structure, occasionally a different conclusion. That variability isn't a defect. It's the property that lets them handle inputs nobody anticipated. But it's a property with a cost, and the cost lands unevenly across a system.
For any component using a model, ask: what happens if this returns something slightly different tomorrow?
Sometimes the honest answer is “nothing much” — a summary reads differently, a draft is worded another way. Sometimes it's “we bill a customer incorrectly,” “we grant the wrong access,” or “our financial reporting doesn't reconcile.” Those belong in different categories, and a surprising number of systems put them in the same one.
Anything computing a charge, applying a discount, or moving funds. These must be reproducible and auditable. A model can help decide which pricing tier applies; the arithmetic itself should never be probabilistic.
Whether a user may see a record is a rules question with a correct answer. “Usually correct” is a security incident with a long tail.
If you have to defend how a number was produced — to an auditor, a regulator, or a court — you need a reproducible path from input to output. “The model determined it” is not a defense.
Anything that must run exactly once, or safely twice. Retries plus non-determinism produces duplicate side effects, and those are miserable to unwind at volume.
The inverse case is equally real, and rules-based approaches fail badly here.
Unstructured to structured. Pulling fields from contracts, emails, or documents where the format varies endlessly. Regex and parsers break on the long tail; models handle it well.
Classification with tolerance. Routing tickets, tagging content, triaging inbound. Occasional misclassification is recoverable, and the alternative is an unmaintainable rules thicket.
Drafting and summarization. First-pass content a human reviews. Variation is fine — sometimes desirable.
Genuinely open-ended reasoning. Problems where enumerating the cases in advance isn't possible. This is the actual frontier capability, and it's used less often than it's invoked.
The strongest production designs rarely choose one or the other. They let the model do what it's good at and put deterministic guardrails around the output.
The model proposes; deterministic code validates. Extract the invoice fields with a model — then verify the line items sum to the stated total with arithmetic. Let the model suggest a pricing tier — then compute the actual charge with the pricing engine. Have it draft the classification — then enforce that the result is one of the permitted values.
This gets you flexibility at the edges and reproducibility where reproducibility matters. It also makes the system testable: you can't exhaustively test a model, but you can exhaustively test the validator.
Teams routinely put a model where a rule belonged — classifying against a fixed list of five values, extracting a field with a stable format, applying logic that has been unchanged for years. Each call costs money and adds latency, and the deterministic version is cheaper, faster, and correct every time.
The reverse also happens: enormous rules engines maintained at great expense to handle variation a model would absorb.
So it's worth asking, feature by feature, why a non-deterministic tool is being used there at all. If the answer is “because we're using AI,” that's not an answer — it's a line item.
Sorting workloads by what they actually require — and pricing each path — is core to how AI operations gets scoped.
See how the Diagnostic works →