
Unwanted variation in judgements that should match, and why organisations ignore it.
Kahneman died 27 March 2024; this was his last book.
Kahneman, Sibony and Sunstein argue that organisations concentrate on bias, meaning systematic error in one direction, and ignore noise, meaning unwanted variability in judgements that should be identical. They show noise is large, expensive and almost entirely unmeasured.
The evidence comes from noise audits. In insurance underwriting, different underwriters priced the same risk with differences averaging around fifty-five percent, far larger than executives predicted. Judges gave different sentences for identical cases, and the same judge gave different sentences depending on time of day, weather and whether the local football team had won. Medical diagnoses, hiring decisions, forensic examinations and forecasts show similar variability.
They decompose noise into level noise, meaning some judges are systematically harsher; pattern noise, meaning individuals respond differently to particular case features; and occasion noise, meaning the same person judges differently on different occasions. Pattern noise is generally the largest component and the least recognised.
The remedies they propose are collectively called decision hygiene: use structured decision protocols that assess independent dimensions separately before aggregating, use relative rather than absolute judgements, aggregate independent judgements from several people before discussion, use simple algorithms or guidelines where available since they eliminate occasion noise entirely, and appoint a decision observer to watch the process.
The book is long and repetitive, and much of it could be stated in half the length. The core empirical finding is well supported and the practical implications for any organisation making repeated judgements are direct.
Executives and investors running any process that makes repeated similar judgements, including hiring, underwriting, credit and investment decisions.
Eugene Higgins Professor of Psychology Emeritus, Princeton University; Nobel laureate in economics 2002
Prospect theory and the heuristics-and-biases programme are the primary psychological foundation the whole of behavioral finance builds on.
“Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein, Noise: A Flaw in Human Judgment (New York: Little, Brown Spark, 2021), 156-158”
Chosen by shared topics, the entries that hold them, and the people who recommended both.
A 20-year test of 28,000 expert forecasts: foxes with many models beat hedgehogs.
“All pundits and commentators should publish their prediction track records, yet don't. What to pay attention to and what to ignore.” — Marc Andreessen
Source: @pmarca, 'Thread: Books I've recently read and recommend'
Recommended by: Marc Andreessen, Michael Mauboussin and 2 others
Why people mistake luck for skill, above all in markets.
“Fooled By Randomness was unquestionably the best non-fiction book I read in 2005.” — Brad Feld
Source: Book Review: Fooled By Randomness — Feld Thoughts
Recommended by: Barton Biggs, Brad Feld, Derek Sivers and 15 others
The research on intuitive judgement and its biases, collected.
“Paul Slovic, Melissa Finucane, Ellen Peters, and Donald G. MacGregor, “The Affect Heuristic,” in Heuristics and Biases: The Psychology of Intuitive Judgment, ed. Thomas Gilovich, Dale Griffin, and Daniel Kahneman (Cambridge: Cambridge University Press, 2002), 397-420” — Michael Mauboussin
Source: Interdisciplinary Perspectives on Risk, page 5
Recommended by: Michael Mauboussin
Thirty-five papers on judgmental heuristics and the biases they produce.
“Research regarding this problem was originally presented in Amos Tversky and Daniel Kahneman, “Evidential Impact of Base Rates,” in Daniel Kahneman, Paul Slovic, and Amos Tversky (eds.) Judgment under Uncertainty: Heuristics and Biases (Cambridge, UK: Cambridge University Press, 1982), 153-160” — Michael Mauboussin
Source: Cultivating Your Judgment Skills: A Framework for Improving the Quality of Decisions, page 13
Recommended by: Michael Mauboussin, Adam Robinson
What a four-year forecasting tournament showed about the best forecasters' habits.
“In a book many in our research department have enjoyed, Superforecasting: The Art and Science of Prediction, Philip Tetlock and Dan Gardner state: “The humility required for good judgment is not self-doubt—the sense that you are untalented, unintelligent, or unworthy.” — Bill Nygren
Source: Bill Nygren Market Commentary | 1Q17
Recommended by: Bill Nygren, Mark Leonard and 11 others
Papers on how people choose under risk, from prospect theory to framing.
As an Amazon Associate I earn from qualifying purchases.