Normal Accidents
Summary
Charles Perrow, a sociologist, wrote this after serving on the inquiry into the Three Mile Island accident, and it argues that some catastrophic failures are structural properties of systems rather than the results of error. His two variables are interactive complexity and tight coupling.
A system is interactively complex when components can affect each other in ways designers did not anticipate and operators cannot see, so that multiple small failures combine into a sequence nobody predicted. A system is tightly coupled when processes are time-dependent, sequences are invariant, and there is no slack that would allow recovery. Systems with both properties, which Perrow says include nuclear power, chemical plants, nuclear weapons and some aircraft and marine operations, will produce what he calls system accidents no matter how well they are managed. Adding safety devices increases complexity and can make things worse, which is why he treats redundancy as an ambiguous remedy.
He works through cases across many industries: Three Mile Island in detail, petrochemical plants, aircraft and air traffic control, marine collisions, dams, mines, space missions and nuclear weapons command and control. His policy conclusion is that some technologies should be abandoned because their accident potential cannot be engineered away, a position that made the book controversial. The main counterargument comes from the high reliability organisations school, which studied aircraft carriers and air traffic control and concluded that organisations can achieve near-error-free operation in complex tightly coupled systems. That debate remains open. The framework has been applied extensively to financial systems since 2008.
Key takeaways 7
- Systems that are both interactively complex and tightly coupled will produce accidents that no amount of management prevents.
- Interactive complexity means failures combine in ways designers did not anticipate and operators cannot diagnose in time.
- Tight coupling removes the slack that would let operators intervene between an initiating failure and the consequence.
- Adding safety systems increases complexity and can create new failure paths, so redundancy is not automatically protective.
- Operator error is usually the label applied after the fact to decisions that were reasonable given the information available.
- Centralised control suits tightly coupled systems while decentralised judgement suits complex ones, and systems with both properties cannot satisfy either requirement.
- Perrow argues that some technologies should be abandoned because their catastrophic potential cannot be engineered away.
Who should read it
Operators and risk managers in complex systems, including financial institutions, who want the structural theory of why catastrophic failures recur despite improved procedures.
Similar books 6
Chosen by shared topics, the entries that hold them, and the people who recommended both.
Why complex systems fail, from Volkswagen to Everest, and how to design against it.
Why large state plans fail when they override local, practical knowledge.
Recommended by: Balaji S. Srinivasan, Nat Eliason
One USB charger followed from a factory in Vietnam by ship and truck to a door.
Incerto volume 4
“Nassim Taleb, author of the new book Antifragile, praises volatility and criticises those who would artificially damp it at great unseen cost. Taleb advises us to be 'anti-fragile'. This is exactly…” — Seth Klarman
Source: Klarman on Taleb's praise of volatility and what Baupost strives for
Recommended by: Mohnish Pabrai, Nat Eliason and 25 others
How complex systems, from subprime lending to space shuttles, drift into failure.
Avoid permanent loss: buy well below value and aim for absolute returns.
“Seth Klarman's 'Margin of Safety' is a good book about risk.” — Arnold Van Den Berg
Source: Van Den Berg on Margin of Safety as a book about risk
Recommended by: Arnold Van Den Berg, Frank Martin, Guy Spier and 6 others
Where it sits in the reference 2
Topics