When to let AI decide, and when to use a rule
If a system can return a different answer to the same input, that's a design choice. The question worth asking of every AI feature is whether anyone actually made it.
Short on time? Book a free 30-minute fit call →
A deterministic tool gives the same output for the same input, every time. A sales-tax calculation, a price list, a late-fee rule, a spreadsheet formula. You can test it exhaustively, reason about it, and reproduce any result later.
A non-deterministic tool may not. Language models are the obvious case — same prompt, different phrasing, different structure, occasionally a different conclusion. That variability isn't a defect. It's the property that lets them handle inputs nobody anticipated. But it's a property with a cost, and the cost lands unevenly across a system.
The question to ask
For any component using a model, ask: what happens if this returns something slightly different tomorrow?
Sometimes the honest answer is “nothing much” — a summary reads differently, a draft is worded another way. Sometimes it's “we bill a customer incorrectly,” “we grant the wrong access,” or “our financial reporting doesn't reconcile.” Those belong in different categories, and a surprising number of systems put them in the same one.
Where determinism is non-negotiable
Anything computing a charge, applying a discount, or moving funds. These must be reproducible and auditable. A model can help decide which pricing tier applies; the arithmetic itself should never be probabilistic.
Whether a user may see a record is a rules question with a correct answer. “Usually correct” is a security incident with a long tail.
If you have to defend how a number was produced — to an auditor, a regulator, or a court — you need a reproducible path from input to output. “The model determined it” is not a defense.
Charging a card, sending an invoice, booking a slot. If a step reruns and the AI answers differently, you get double charges that are miserable to unwind.
Where non-determinism earns its place
The inverse case is equally real, and rules-based approaches fail badly here.
The pattern that works: propose, then verify
The strongest production designs rarely choose one or the other. They let the model do what it's good at and put deterministic guardrails around the output.
The model proposes; deterministic code validates. Extract the invoice fields with a model — then verify the line items sum to the stated total with arithmetic. Let the model suggest a pricing tier — then compute the actual charge with the pricing engine. Have it draft the classification — then enforce that the result is one of the permitted values.
This gets you flexibility at the edges and reproducibility where reproducibility matters. It also makes the system testable: you can't exhaustively test a model, but you can exhaustively test the validator.
Where AI gets overused
A common mistake is using AI where a rule would do: sorting email into five fixed folders, reading a date from a form that never changes. A rule is cheaper, faster and right every time.
The reverse also happens: a pile of hand-built rules that breaks every time a customer phrases something new.
So it's worth asking, feature by feature, why a non-deterministic tool is being used there at all. If the answer is “because we're using AI,” that's not an answer.
Want this fixed in your business?
Book a free 30-minute fit call. Tell me where the work is getting away from you, and I'll tell you straight whether it's worth fixing.