Genius Makers
How deep learning found the GPU, the demand Nvidia was built on.
Summary
Metz, a New York Times technology reporter, tells the history of deep learning through its people: Geoffrey Hinton, Yann LeCun and Yoshua Bengio through the decades when neural networks were dismissed, then the 2012 ImageNet result, the auction of Hinton's DNNresearch to Google, the founding and sale of DeepMind, the creation of OpenAI, and the competition for researchers among Google, Facebook, Microsoft, Baidu and the startups.
The substance is the recruitment war and what it reveals. Metz documents specific numbers, including the auction that took Hinton's three-person company to forty-four million dollars, and the compensation levels that turned a small academic subfield into the most expensive labour market in technology within about three years. He shows that the technical breakthrough was not new theory but the combination of old algorithms with graphics processing units and large labelled datasets, and that the researchers who persisted through the years of rejection did so on conviction rather than evidence.
He also covers the ethical arguments as they arose inside the companies: Google's Project Maven and the employee revolt, facial recognition bias documented by Timnit Gebru and Joy Buolamwini, and the internal disagreements about deployment.
The book is reported journalism and does not explain the technology in depth; readers wanting that should look elsewhere. It ends in 2021 and predates the large language model era, so the strategic picture is dated, though the account of how the field's labour market and corporate concentration formed remains the best available.
Key takeaways 7
- Deep learning's breakthrough came from combining decades-old algorithms with graphics processing units and large labelled datasets, not from new theory.
- The 2012 ImageNet result converted a dismissed subfield into the dominant approach in about eighteen months.
- Hinton auctioned his three-person research company to Google for forty-four million dollars, which set the price for machine learning researchers across the industry.
- A handful of large companies acquired essentially the entire population of senior researchers in the field within a few years, which concentrated capability permanently.
- The researchers who persisted with neural networks through the decades of rejection did so on conviction, since the evidence supporting them did not exist until the compute did.
- Corporate research labs took over from universities as the location of frontier work because only they could supply the compute and data required.
- Internal employee objection, rather than regulation or external pressure, was the mechanism that changed several major companies' deployment decisions.
Who should read it
Investors and founders who want the reported history of how deep learning's talent and capability concentrated in a few firms.
Similar books 6
Chosen by shared topics, the entries that hold them, and the people who recommended both.
The OpenAI-DeepMind race that set the shape of Nvidia's customer base.
Recommended by: Andrew Wilkinson
The founder scene around OpenAI, reported from inside the funding rounds.
The rationalists around LessWrong and their fear of artificial intelligence risk.
The struggle to make machine learning systems do what people actually intend.
Recommended by: Sam Altman
Why AI and synthetic biology will spread fast and cheaply, and how to contain them.
“Everything is about to be changed by AI in a radical, scary way” — Andrew Wilkinson
Source: Newsletter
Recommended by: Andrew Wilkinson, Bill Gates and 1 other
Where it sits in the reference 4
Topics