
Why AI must be redesigned from the ground up to keep superhuman machines in check.
Control reframed as uncertainty about human objectives.
Russell, an AI researcher, argues that the standard model of the field, in which a machine optimises a fixed objective that humans specify, is dangerously misguided. A specified goal never captures everything people value, and a superhuman system pursuing it could be catastrophic. Because no one knows when human-level AI will arrive, or how long safety research will take, he argues that work on control should start now. Economic pressure makes further progress certain, since human-level AI could be worth trillions of dollars.
The book first examines intelligence in humans and machines, the near-term benefits of AI, the breakthroughs still needed, and current misuses such as lethal autonomous weapons. It answers common arguments that dismiss AI risk. Russell then proposes three principles: the machine's only objective is to realise human preferences, it is initially uncertain what those preferences are, and human behaviour is the source of information about them. He suggests inverse reinforcement learning as a mechanism and closes by calling for tighter governance. Reviewers in Nature and The New York Times doubted that superintelligence is possible.
Investors, policymakers and engineers who want the case for AI risk and a concrete proposal for control from a leading AI researcher.
Chosen by shared topics, the entries that hold them, and the people who recommended both.
How AI could change crime, war, justice, jobs and what it means to be human.
“A deep, bold, and visionary dive into Artificial Intelligence and its many implications. One of the most interesting books I've ever read. If you haven't read much of AI yet, start with “Surviving AI” as an intro, then read this as a deep-dive. His perspective is amazingly thorough. Defining terminology was a great way to start. For example life is a “process that can retain its complexity and replicate”. Intelligence is the “ability to accomplish complex goals”. That keeps it broad enough to define future technology as alive and intelligent.” — Derek Sivers
Source: Derek Sivers book notes (primary, rated list) · Rating: 7/10
Recommended by: Andrew Wilkinson, Derek Sivers, Bill Gates and 3 others
Why machines smarter than humans pose a control problem to solve before they arrive.
“I've read a few of the books that get tossed around, like Bostrom's Superintelligence, and a bunch of the articles that people have written.” — Brad Feld
Source: What Is The "Third Wave" Of This Generation? — Feld Thoughts
Recommended by: Brad Feld, Michael Mauboussin and 9 others
Recommended by: Patrick Collison, Cynthia Johnson, Elon Musk
The textbook on deep learning: how machines learn layered concepts from data.
Recommended by: Vinod Khosla, Elon Musk and 2 others
What humanity may seek after famine, plague and war: immortality and divinity.
“Very interesting alternative perspective on life from a historian. Anti-religion, anti-humanism, pro-animal. Seems detached, but is quite opinionated. Much to think about, regardless. My notes here give a taste. A lot to think about.” — Derek Sivers
Source: Derek Sivers book notes (primary, rated list) · Rating: 6/10
Recommended by: Andrew Wilkinson, Bill Gates, Derek Sivers and 8 others
The benefits and risks of human-level and superhuman artificial intelligence.
Recommended by: Elon Musk
As an Amazon Associate I earn from qualifying purchases.