Why AI without evidence creates expensive mistakes

AI is only ever as good as what it is standing on.

Give a modern model a clean, well-labelled set of facts and it is genuinely brilliant. Give it a pile of assumptions dressed up as facts and it is still brilliant, in the worst possible way: fast, fluent, completely sure of itself, and wrong. It will not hesitate. It has no way of knowing that half its inputs were somebody’s best guess from a spreadsheet in 2023.

This is the part of the AI conversation I think we are getting wrong. Everyone is arguing about which model is smartest. In a real business, the model is rarely the weak link. The inputs are.

Fluent is not the same as right

A human expert hedges. When you ask an operations director whether a site can absorb another two hundred seats, they answer in texture: probably, but the attrition number they are using is soft. The hedge is information. It tells you where to push.

A model strips the hedge out. Ask it the same question and you get a crisp paragraph with a number in bold. The uncertainty that lived in the expert’s tone has been quietly deleted, and what is left reads like fact. That is not intelligence. That is faster guessing with better grammar.

And a wrong answer delivered with total confidence is worse than no answer at all. No answer makes you go and look. A confident wrong answer makes you sign.

Label the inputs, not just the outputs

The fix is not a smarter model. It is knowing, for every input, whether it was measured, modelled, inferred, merely stated by someone, or never measured at all.

That is the whole idea behind the evidence hierarchy. Measured beats modelled. Modelled beats inferred. Inferred beats stated. Stated beats unmeasured. Every fact carries its own quality state, so when AI reasons over them it can weight a hard measurement above a hopeful guess, and it can tell you, out loud, which of the two it leaned on.

That single move changes the character of the answer. Instead of here is what to do, you get here is what to do, and here is the one input I am least sure about. Now you know exactly where to spend your scepticism.

Whose model, on whose evidence

It also matters that the reasoning runs on evidence you control. That is why we let enterprises run this on their own model contracts and data terms rather than pouring their facts into someone else’s black box; the short version of that is BYO AI. Your evidence, your governance, your model.

Point a confident machine at unlabelled assumptions and it will make expensive mistakes at a speed no human could match. Point it at evidence that knows its own quality, and it becomes something worth trusting. The model was never the hard part. The ground it stands on is.