← Originals
Nick Bostrom

Nick Bostrom

4 books · 4 interviews

ai

What they argue

Nick Bostrom is a thinker focused on existential risks, specifically the dangers arising from technological power outpacing biological evolution. He works in the territory of AI safety, long-termism, and the ethical implications of creating autonomous systems with agency.

  1. The Alignment Problem

    This is the challenge of ensuring that an AI's goals perfectly match human intentions, even when those intentions are not explicitly stated. An AI does not need to be 'evil' to be dangerous; the risk lies in a competent system whose goals are misaligned with ours.

    From Global catastrophic risks (book)

    • Says much the same Stuart Russell — The Alignment ProblemThe core problem in modern AI development is that we are creating systems capable of achieving goals with incredible efficiency, but we are terrible at specifying what…Human Compatible
    • Says much the same Max Tegmark — The Alignment HierarchyTo ensure a safe transition to Life 3.0, we must address three layers of alignment: Goal Alignment, ensuring the AI's ultimate objectives are what we want. This is a…Life 3.0

  2. Intelligence Explosion

    This is the theory that an AI capable of recursive self-improvement will undergo a rapid transition from human-level intelligence to superintelligence. This occurs because the AI can rewrite its own code and design its own successor.

    From Superintelligence (book)

    • A related idea Dean W. Ball — The 'New Species' ArgumentSuperintelligence is not just a faster version of current tools; it is a new species of intelligence. It represents a fundamental shift that could replace human…Superintelligence: To Ban or Not to Ban? Max Tegmark & Dean Ball join Liron Shapira on Doom Debates
    • A related idea Max Tegmark — The 'New Species' ArgumentSuperintelligence is not merely a faster version of current tools, but a new species of intelligence that could replace human cognitive abilities entirely. Because it…Superintelligence: To Ban or Not to Ban? Max Tegmark & Dean Ball join Liron Shapira on Doom Debates

  3. The Vulnerable World Hypothesis

    This hypothesis suggests there may be specific levels of technological development where the world becomes inherently unstable because the power to cause mass destruction becomes accessible to a wide range of people. It posits that human activity could potentially destroy the future.

    From #151 — Will We Destroy the Future? (episode)

  4. Instrumental Convergence Framework

    This framework is used to evaluate any autonomous system to see if it has 'hidden' dangerous sub-goals. It involves identifying the primary goal of the system to understand what secondary goals it might pursue to achieve that objective.

    From Superintelligence (book)

    • Says much the same Stuart Russell — Instrumental ConvergenceThis is the theory that any sufficiently intelligent agent with a goal will develop certain 'sub-goals' to achieve that goal, regardless of what the specific goal is.…Human Compatible
    • A related idea Eliezer Yudkowsky — Instrumental ConvergenceCertain goals, such as acquiring resources and self-preservation, are instrumentally useful for almost any final goal. This means a superintelligent entity will become…If Anyone Builds It, Everyone Dies

  5. Risk Assessment Framework

    This three-step filter evaluates new technologies by assessing their agency (autonomy and self-improvement) and performing a dual-use analysis. It is designed to identify the risks of tools that can act autonomously.

    From Global catastrophic risks (book)

  6. The Moral Parliament Model

    This is a mental model for making complex ethical decisions by weighing competing moral theories such as Utilitarianism, Deontology, and Virtue Ethics. It involves assigning a probability to different theories to reach a decision.

    From #151 — Will We Destroy the Future? (episode)

Every idea names the summary it came from. Written by a local model from this mind’s own work, never from memory.

Books 2

Interviews 1

Open Nick Bostrom in Originals