Mustafa Suleyman — Can We Contain Artificial Intelligence?

In one sentence

The central challenge of the 21st century is not just the technical achievement of creating "superintelligence," but the systemic problem of how to contain and govern the massive proliferation of power inherent in general-purpose technologies like AI and synthetic biology.

Why read this (and who it's for)

This conversation addresses the "containment problem"—the difficulty of keeping a rapidly evolving, general-purpose technology from causing systemic harm. It is essential for anyone interested in AI safety, policy, and the societal shifts caused by automation. You will learn to distinguish between "superintelligence" hype and the practical, near-term risks of labor disruption, misinformation, and asymmetric threats.

The big idea

Mustafa Suleyman argues that we are entering a period of unprecedented technological proliferation. He defines this as a "Coming Wave"—a general-purpose technology (like fire, electricity, or the steam engine) that acts as an enabler for other innovations. Unlike a niche tool, a "wave" technology becomes cheaper, more accessible, and more integrated into the human ecosystem over time.

The core of Suleyman's concern is that the sheer scale of these technologies—specifically their ability to generate new knowledge, automate high-level cognitive labor, and manipulate biological systems—creates a power imbalance. Because these technologies are "general purpose," they can be used for both immense good (curing diseases, boosting productivity) and immense harm (bioweapons, misinformation, state-sponsored surveillance). The danger isn't just a "rogue AI" from a sci-fi movie; it is the systemic inability of our current social and regulatory institutions to contain a technology that scales exponentially while our governance remains linear.

Key ideas

The "Coming Wave" of General Purpose Technology

• What it is — A technology that serves as a foundational enabler for other technologies (e.g., electricity). It proliferates by becoming cheaper, easier to use, and more integrated into daily life.
• Why it matters — Because these technologies are "general," they don't just solve one problem; they change the entire landscape of what is possible, creating a "Cambrian explosion" of new capabilities and risks.
• In the book — Suleyman compares the current AI/biotech wave to historical waves like fire and the wheel, noting that while they were almost universally good, they fundamentally transformed the human species.

The Distinction Between Superintelligence and Practical Risk

• What it is — Suleyman argues that the "superintelligence" narrative (AI recursively self-improving until it rules the world) is a distraction from more immediate, practical risks.
• Why it matters — By focusing only on "doomsday" scenarios, we ignore the very real, near-term harms of labor displacement, misinformation, and the erosion of human agency.
• In the book — He notes that while he cares about existential risk, the "practical risks" of the next 10–20 years are what require urgent, actionable policy and safety measures.

The Containment Problem

• What it is — The challenge of ensuring that a powerful technology remains under human control and is used for the public good rather than for malicious, asymmetric, or destabilizing purposes.
• Why it matters — As AI becomes "smaller" (cheaper to run) and more capable, it becomes harder to gatekeep. If we cannot contain the application of the technology, we lose the ability to manage its impact on society.
• In the book — Suleyman highlights that we are already in a "wild" environment where open-source models are proliferating, making the "digital oracle in a box" (an air-gapped, controlled AI) a harder-to-achieve goal.

The Proliferation of Intelligence

• What it is — The idea that intelligence—the ability to "get stuff done"—is becoming a commodity that can be scaled via compute and data.
• Why it matters — This creates a "meritocratic" moment where anyone with a smartphone can access elite-level education, medical advice, and personal assistance, but it also threatens to disrupt the very foundations of the labor market.
• In the book — He describes the "Cambrian explosion of productivity" that will occur when billions of people gain access to high-level cognitive assistants.

The Misinformation Apocalypse

• What it is — The risk of AI-generated content making it impossible to discern truth from falsehood, leading to a breakdown in social trust and shared reality.
• Why it matters — If we cannot verify what is real, the "machinery of statecraft" and the "public square" both become vulnerable to manipulation.
• In the book — Suleyman discusses the need for "digital watermarks" and other friction-based measures to slow the deployment of harmful synthetic content.

Frameworks & tools

The "Coming Wave" Analysis

To evaluate any new technology's potential impact, use this framework:

1. Identify the General Purpose Nature: Does this technology enable other technologies? (e.g., Is it a tool or a platform?)
2. Assess the Proliferation Curve: How quickly is it becoming cheaper and more accessible?
3. Determine the "Power" Shift: Who gains the ability to "get stuff done" with this tool? Does it create a monopoly or a democratization of power?
4. Identify the "Downsides" of Success: If this technology succeeds perfectly (e.g., everyone has an AI tutor), what are the secondary effects (e.g., labor displacement, loss of human skill)?

The Containment Checklist

When considering the deployment of a new AI or biotech capability, ask:

1. Is it "Air-Gapped"? Can the technology be run without an internet connection?
2. Is there "Friction"? Are there intentional barriers to prevent rapid, unmonitored scaling?
3. Who is the "Accountable" Actor? Is there a clear entity responsible for the safety and ethics of the model?
4. What is the "Asymmetric Threat"? Can this be used by a small group to cause massive, disproportionate harm (e.g., a lone actor creating a pathogen)?

Memorable examples & stories

The AlphaGo "Mistake"

During the famous AlphaGo match, a 9-dan expert commentator thought the AI made a brilliant move, only to realize later it was a "mistake" that turned out to be a novel strategy. Lesson: AI can discover "new knowledge" that is fundamentally outside the human experience of the game.

The "Pacific Ocean" of Calculations

Suleyman describes the scale of modern LLMs by comparing the number of floating-point operations they perform to the amount of water in the Pacific Ocean. Lesson: The sheer scale of compute is a "force multiplier" that makes the quality of the algorithm even more significant.

The "200 Million PhD Theses"

Suleyman notes that AlphaFold accomplished the equivalent of 200 million PhD theses in a few years regarding protein folding. Lesson: Technology doesn't just do things faster; it compresses time and effort in ways that are almost impossible for humans to replicate.

Key quotes

"I think that the language around superintelligence has actually been a bit of a distraction... I think there's a bunch of more near-term, very practical things that we should be concerned about." Why it matters: It shifts the focus from sci-fi "paperclip" scenarios to the immediate dangers of labor disruption and misinformation.

"The core emphasis was that intelligence was about generality... intelligence is the ability to perform well across a wide range of environments." Why it matters: This explains why AI is a "general purpose" technology rather than a niche tool.

"We have to start with our best assessment of the facts and try to reject those facts if they're, you know, inaccurate in some way, and then try to collectively predict what the consequences are going to be like." Why it matters: It calls for a move away from "pessimism aversion" toward rigorous, evidence-based forecasting.

How to apply it

This week

• Audit your "Intelligence" tools: Identify which parts of your work are "higher cognitive" (judgment, creativity, empathy) and which are "generative" (summarizing, drafting, organizing).
• Identify the "Coming Waves" in your industry: What general-purpose technologies are currently making your specific job easier or more accessible?
• Practice "Pessimism-Free" Analysis: When discussing a new technology, explicitly list three "downside" risks that are not "doomsday" scenarios (e.g., "How will this affect the entry-level job market?").

Ongoing

• Monitor "Friction" in AI: When using tools like ChatGPT or Claude, ask: "What is the cost of this? Who is being displaced by this? Is there a watermark?"
• Prioritize "Human-Only" Skills: Focus on developing the skills AI cannot easily replicate: deep empathy, complex human judgment, and high-stakes accountability.

Common pitfalls

• The "Superintelligence" Trap: Many people get so caught up in the "AI will become God" narrative that they ignore the fact that AI is already being used to automate jobs, create deepfakes, and influence elections today.
• The "Optimism/Pessimism" Binary: Suleyman warns against viewing these as moral stances. They are both biases. The goal is to be accurate about the facts, regardless of whether the outcome feels "good" or "bad."
• Underestimating "Scale": People often underestimate how much "more compute" and "more data" can fundamentally change the capabilities of a model.

Connects to

• The Alignment Problem (Brian Christian): Explores the technical side of ensuring AI goals match human values.
• The Age of AI (Kissimmee, et al.): Provides a broader historical context for the "waves" of technology.
• Superintelligence (Nick Bostrom): The "superintelligence" framework that Suleyman argues is a distraction from near-term practical risks.

If you remember only three things

• The "Coming Wave": AI and synthetic biology are general-purpose technologies that will proliferate, become cheaper, and become ubiquitous.
• The Containment Problem: Our primary challenge is not just "making AI smart," but managing the risks of its proliferation across labor, information, and biology.
• Practicality over Hype: Focus on the immediate risks of labor disruption and misinformation rather than just the theoretical risks of a "superintelligence" takeover.

Internalize it

• Reflection questions
  • If a tool can perform 80% of your current cognitive tasks, what is the remaining 20% that provides your unique value?
  • How would your industry change if the "cost of intelligence" dropped to near-zero tomorrow?
  • What "friction" would you personally be willing to accept (e.g., slower processing, higher costs) to ensure a technology is being developed safely?
• Flashcards Q: What is a "Coming Wave" technology? A: A general-purpose technology (like electricity) that enables other technologies, proliferates by becoming cheaper/more accessible, and transforms the human ecosystem. Q: Why does Suleyman think "superintelligence" is a distraction? A: Because it draws attention away from near-term, practical risks like labor disruption, misinformation, and the erosion of human agency. Q: What is the "Containment Problem"? A: The challenge of ensuring that powerful, general-purpose technologies are used for the public good and do not cause systemic, unmanageable harm. Q: How does "Scale" affect AI? A: Because the algorithms are general, they can be "turbocharged" by traditional computing assets like Moore's Law, perfect memory, and parallelizable compute. Q: What is the "Cambrian Explosion of Productivity"? A: The period where everyone gains access to tools that allow them to pursue their own agendas, making everyone smarter and more productive.

Fidelity note

This summary is grounded in the provided transcript of the conversation between Sam Harris and Mustafa Suleyman. I am confident in the definitions of "Coming Wave" and the "Containment Problem" as they are central to Suleyman's book. The distinction between "superintelligence" and "practical risk" is a key nuance of the conversation. The transcript does not provide a specific, step-by-step "how-to" for containment, so the "Frameworks & tools" section is an extrapolation of Suleyman's arguments into actionable checklists.