Decision Quality vs Outcome Quality

The most natural way to judge a decision is by how it turned out. It is also the most corrosive habit an organisation can teach itself, because it hands the verdict to the one juror who was never in the room: luck.

Decision quality and outcome quality are different things. Decision quality is the soundness of the reasoning at the moment of choice: the evidence gathered and honestly weighed, the alternatives genuinely considered, the risk sized, all judged against what was knowable at the time. Outcome quality is how the story ended. Between the two sits everything the decider could not control. The separation is the founding insight of decision analysis: Ronald Howard, who coined the field’s name in the 1960s, insisted that a decision be judged apart from its outcome, and the Decision Quality framework developed by Carl Spetzler and his colleagues built a whole practice on that insistence. Psychology later named the failure to separate them: Jonathan Baron and John Hershey called it outcome bias, the tendency to rate identical decisions differently once you know how things turned out. Poker supplied the bluntest word of all: Annie Duke, in Thinking in Bets, calls judging decisions purely by results “resulting”, and treats it as the amateur’s defining mistake.

A decision should be judged by the reasoning at the time, against what was knowable at the time. Outcomes are evidence about processes, not verdicts on single decisions.

A sound decision can lose

Put decision quality on one axis and outcome quality on the other, and every decision lands in one of four cells. Good decision, good outcome: the deserved win, pleasant and uninformative. Bad decision, bad outcome: the deserved loss, painful and equally uninformative. The learning lives in the other two cells. Good decision, bad outcome: the well-reasoned bet that lost to chance. Bad decision, good outcome: the reckless call rescued by it. An organisation that reads only the outcome column punishes the third cell and promotes the fourth, which means it systematically fires its discipline and funds its luck.

What resulting teaches your best people

People learn what reviews reward faster than any policy can announce it. If the unlucky are punished, the careful learn to avoid any bet whose downside is visible and attributable to them. They start preferring the defensible decision to the correct one. They herd, because a shared failure is survivable and a solitary one is not. They write their records to survive hindsight rather than to inform the next decision. None of this appears in any values statement, and all of it follows, quite rationally, from outcome-only judgement. The organisation slowly selects against exactly what it most needs: bold, careful bets under genuine uncertainty, made by people who trust that the reasoning will be examined fairly if the world misbehaves.

What to review instead

Review the reasoning at the time, against what was knowable at the time. In practice that means asking, of any consequential decision:

• Was the evidence gathered, and was its quality honestly labelled: what was measured, what was modelled, what was merely assumed?
• Were real alternatives priced, or was one option dressed up for the meeting?
• Was the binding constraint identified, and was the downside sized and survivable?
• Were the choice, the owner and the expected outcome recorded when the decision was made, not reconstructed afterwards?

Then, separately, let outcomes do the work they are actually fit for. One outcome says little about one decision; over enough repeated decisions, luck washes out, and patterns of outcomes become legitimate evidence about the process that produced them. Judge single decisions by process. Judge processes by accumulated outcomes.

One concrete example

Clearly illustrative, with no customer implied. Two account directors face expansion decisions in the same quarter. The first builds the careful case: evidence labelled by quality, the downside sized, an explicit walk-away point, the reasoning recorded at the time. Months later a change of control on the client side, unknowable when the decision was made, kills the deal. The second accepts a vaguely scoped expansion against internal advice, keeps no record, and is rescued by a market swing that lifts the client’s budget. The outcome-only review cautions the first director and celebrates the second. Everyone watching draws the intended conclusion: do not make careful bets, make lucky ones, and above all make popular ones. A process review would have reached the opposite verdict. The first decision deserves repeating, and the second deserves alarm, because luck is not a method.

Keeping “what was knowable” honest

The honest objection to judging decisions by their reasoning is that hindsight corrupts the evidence: by review time, memory has quietly backfilled what everyone supposedly knew. That is a records problem before it is a virtue problem, and it is the problem decision intelligence is built to remove. A scenario run is an immutable, timestamped pricing of the options against the facts as they stood, so what was knowable at the time survives as a record rather than a recollection, with every fact carrying its evidence state. The decision itself is recorded with its owner, and the eventual result is scored in an Outcomes Ledger against both what was recommended and what was decided, which is precisely the information needed to tell bad luck from bad process. And because the learning loop mines patterns only once enough scored outcomes accumulate, outcomes are used the way they deserve to be: in aggregate, as evidence about the process, never as a verdict on the individual who made one honest bet that lost.

Organisations do not get to choose their outcomes. They do get to choose their reasoning, their records and their reviews. Judge those, and you are judging the only things a decider ever controlled.