
Why some forecasts succeed and most fail, from weather to earthquakes and economics.
Silver examines forecasting across domains where records exist, and argues that the difference between good and bad forecasting is a specific set of practices rather than better models or more data.
The domains he covers include weather forecasting, which he identifies as one of the genuine success stories with measurable improvement over decades; earthquakes, where he argues prediction is not currently possible and the attempts have produced false confidence; economics, where he documents that forecasters do not publish confidence intervals and that the intervals they do have are far wider than users assume; baseball, poker, chess, political polling, epidemiology and climate.
His central prescription is Bayesian reasoning, meaning starting from a prior probability, updating it in proportion to how diagnostic new evidence is, and stating beliefs as probabilities rather than as positions. He argues that the failure to state a prior is what makes most forecasting unfalsifiable and unimprovable. His analysis of the 2008 crisis identifies four specific failure points: the rating agencies' models assumed regional rather than national default correlation, the housing boom was treated as sustainable, the effects of the collapse were underestimated, and the models were built on data from a period containing no comparable event.
The book was published in 2012 and Silver's own subsequent forecasting record has been mixed, particularly around 2016, which he addressed by noting his model gave a higher probability than others. The critique that Bayesian methods depend on the prior, which is chosen rather than derived, applies to his prescription. The domain comparisons are the most valuable part.
Anyone who makes or consumes forecasts and wants a comparison of which domains forecast well and what the successful ones do differently.
“See Gigerenzer, Calculated Risks, 28-29. 14 For a book that treats this topic and more, see Annie Duke, Thinking in Bets: Making Smarter Decisions When You Don't Have All the Facts (New York: Portfolio/Penguin, 2018). 15 For a good discussion of Bayes’ Theorem, see Nate Silver, The Signal and the Noise: Why So Many Predictions Fail—But Some Don’t (New York, The Penguin Press, 2012), 232-261.”
Chosen by shared topics, the entries that hold them, and the people who recommended both.
A 20-year test of 28,000 expert forecasts: foxes with many models beat hedgehogs.
“All pundits and commentators should publish their prediction track records, yet don't. What to pay attention to and what to ignore.” — Marc Andreessen
Source: @pmarca, 'Thread: Books I've recently read and recommend'
Recommended by: Marc Andreessen, Michael Mauboussin and 2 others
What a four-year forecasting tournament showed about the best forecasters' habits.
“In a book many in our research department have enjoyed, Superforecasting: The Art and Science of Prediction, Philip Tetlock and Dan Gardner state: “The humility required for good judgment is not self-doubt—the sense that you are untalented, unintelligent, or unworthy.” — Bill Nygren
Source: Bill Nygren Market Commentary | 1Q17
Recommended by: Bill Nygren, Mark Leonard and 11 others
Why anything a manager cares about can be measured, and how to measure it.
“Hubbard, Douglas, How to Measure Anything: Finding the Value of Intangibles in Business (Hoboken, NJ: John Wiley & Sons, 2007)” — Michael Mauboussin
Source: The Wisdom of Crowds in Markets: Crowd Behavior in Prediction, Betting, and Stock Markets
Recommended by: Michael Mauboussin, Julia Galef, Nick Ganju
Unwanted variation in judgements that should match, and why organisations ignore it.
“Daniel Kahneman, Olivier Sibony, and Cass R. Sunstein, Noise: A Flaw in Human Judgment (New York: Little, Brown Spark, 2021), 156-158” — Michael Mauboussin
Source: The Impact of Intangibles on Base Rates, page 10
Recommended by: Michael Mauboussin
Whether clinical judgement or statistical formulas predict better in psychology.
“Paul E. Meehl, Clinical versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence (Minneapolis, MN: University of Minnesota Press, 1954)” — Michael Mauboussin
Source: BIN There, Done That: How to Reduce the Sources of Forecasting Error, page 14
Recommended by: Michael Mauboussin, Daniel Kahneman
How randomness shapes our lives, the history of probability, and why we misread it.
“I thought it was about the philosophy of randomness, but turned out to be about the math of probability. Might read again some day when in the mood for that.” — Derek Sivers
Source: Derek Sivers book notes (primary, rated list) · Rating: 3/10
Recommended by: Derek Sivers, Michael Mauboussin and 1 other
As an Amazon Associate I earn from qualifying purchases.