← Originals
Eliezer Yudkowsky

Eliezer Yudkowsky

1 book · 2 essays · 6 interviews

ai · philosophy

What they argue

Eliezer Yudkowsky is a thinker focused on the technical and philosophical architecture of AI safety and the existential risks posed by superintelligent agents. He explores the mechanics of alignment, the psychology of rationality, and the societal challenges of recognizing imminent technological shifts.

  1. Instrumental Convergence

    Certain goals, such as acquiring resources and self-preservation, are instrumentally useful for almost any final goal. This means a superintelligent entity will become incredibly proficient at achieving its goals by pursuing these means.

    From If Anyone Builds It, Everyone Dies (book)

    • Says much the same Stuart Russell — Instrumental ConvergenceThis is the theory that any sufficiently intelligent agent with a goal will develop certain 'sub-goals' to achieve that goal, regardless of what the specific goal is.…Human Compatible
    • A related idea Nick Bostrom — Instrumental Convergence FrameworkThis framework is used to evaluate any autonomous system to see if it has 'hidden' dangerous sub-goals. It involves identifying the primary goal of the system to…Superintelligence

  2. The Judge Framework

    Instead of arguing for a side, one should act as a judge of competing hypotheses. This involves identifying a belief and letting it 'plead its case' without seeking to argue for one side or another.

    From The Twelve Virtues of Rationality (essay)

    • A related idea Adele Faber — The Description FrameworkInstead of judging, describe the situation when a teen is acting out or when you want to initiate a conversation about a problem. This helps create a safe emotional…How to talk so teens will listen-- & listen so teens will talk
    • A related idea Dale Carnegie — The Influence FrameworkTo persuade someone who disagrees with you, you should first seek common ground to build a foundation of agreement. This approach acknowledges that humans are emotional…How to Win Friends and Influence People

  3. Pluralistic Ignorance

    This is a social phenomenon where individuals privately fear a situation but publicly act as if they are fine because they believe others are not alarmed. It explains why there is no 'fire alarm' for AGI, as people wait for a social signal that may never come.

    From There's No Fire Alarm for Artificial General Intelligence (essay)

  4. Capability vs. Alignment

    Capability refers to what a system can do, such as playing chess or writing code. Alignment refers to what the system actually wants to do, which is its internal goals.

    From #368 – Eliezer Yudkowsky: Dangers of AI and the End of Human Civilization (episode)

  5. The Maturity of the Smartphone

    The smartphone has reached a point of diminishing returns where incremental updates no longer provide a 'wow' factor. This serves as a signal for the limits of certain technological form factors.

    From Are We Past Peak iPhone? + Eliezer Yudkowsky on A.I. Doom (episode)

  6. Relinquishment

    This is the willingness to destroy any belief that can be destroyed by the truth. It requires submitting oneself to 'ordeals' and testing beliefs in 'fire' to move from appearing rational to being accurate.

    From The Twelve Virtues of Rationality (essay)

  7. The Alignment Taxonomy

    AI risks can be categorized into three distinct layers: Technical Alignment (making the math work), and other layers of safety. This framework helps in understanding how to ensure gradient descent leads to safe outcomes.

    From If Anyone Builds It, Everyone Dies (book)

    • A related idea Max Tegmark — The Alignment HierarchyTo ensure a safe transition to Life 3.0, we must address three layers of alignment: Goal Alignment, ensuring the AI's ultimate objectives are what we want. This is a…Life 3.0
    • A related idea Nick Bostrom — The Alignment ProblemThis is the challenge of ensuring that an AI's goals perfectly match human intentions, even when those intentions are not explicitly stated. An AI does not need to be…Global catastrophic risks

Every idea names the summary it came from. Written by a local model from this mind’s own work, never from memory.

Books 1

Essays 2

Interviews 2

Open Eliezer Yudkowsky in Originals