We have entered the 'middle game' of AI, where recursive self-improvement is beginning to manifest, and the primary challenge is navigating the geopolitical and safety implications of a world where human-led research may soon be outpaced by machine-driven iteration.
In one sentence
The AI trajectory has shifted from the "beginning" (initial breakthroughs) to the "middle" (recursive self-improvement and systemic integration), where the core challenge is maintaining human agency over a process that is becoming increasingly autonomous.
Why read this (and who it's for)
This conversation is for AI researchers, policy-makers, and "pilled" enthusiasts who want to move past the hype of "GPT-5 is coming" and into the structural reality of the AI "middle game." It solves the confusion of why progress feels both slower and faster simultaneously by identifying the shift toward recursive self-improvement. You will be able to identify the "live players" in the current landscape, understand the risks of "vibe-coded" safety, and recognize the dangers of "forbidden" research techniques.
The big idea
Zvi Mowshowitz argues that we are currently in the "middle game" of the AI story. While many believe we are approaching an S-curve plateau where intelligence will be commoditized and life will remain fundamentally "normal," Zvi contends that we are actually entering a period of recursive self-improvement. In this phase, AI begins to drive its own advancement, creating a feedback loop where human research talent becomes less of a bottleneck.
The logic of the "middle game" is that while we aren't seeing "God-like" transformations in every household yet, the underlying mechanics—faster cycles, labs pulling away from one another, and humans being removed from the loop—are the precursors to the "end game." The end game, Zvi defines as the point where human research talent no longer matters because the AI is the primary driver of its own evolution.
Key ideas
The "Middle Game" vs. The "End Game"
- What it is — The "middle game" is the current era where AI-driven cycles are accelerating, and labs are beginning to recognize different services and capabilities. The "end game" is the point where AI-driven recursive self-improvement makes human research talent irrelevant.
- Why it matters — Recognizing this distinction prevents us from falling for the "S-curve" trap, where people assume progress will plateau because they want to believe the future will be "normal."
- In the book — Zvi notes that people "desperately want to tell themselves a story" where AI is just "Internet-big" because they want to stay sane and plan for an ordinary human future.
Recursive Self-Improvement (RSI)
- What it is — The process where AI systems are used to improve the underlying architecture or capabilities of the next iteration of AI, creating a compounding effect on intelligence.
- Why it matters — Once this dynamic hits a "point of no return," the speed of improvement may exceed human ability to monitor or intervene.
- In the book — Zvi argues we are currently seeing "steadily increasing amounts of the [AI] improvement" and that we are entering the period where this dynamic becomes the primary driver of progress.
The "Live Players" and Laggards
- What it is — A ranking of the current leaders in the frontier model race. Zvi identifies Anthropic and OpenAI as the leaders, with Google currently at risk of falling out of the top tier.
- Why it matters — Understanding who holds the "keys" to the most capable models determines who sets the safety standards and who controls the most powerful tools.
- In the book — Zvi argues that Meta is a "consumer company" and that Zuckerberg's vision of "super intelligence" might actually just mean "super selling ads."
The Fallacy of "Vibe-Based" Safety
- What it is — The critique of safety measures (like Anthropic's Constitutional AI) that rely on "vibes" or high-level principles rather than rigorous, verifiable technical constraints.
- Why it matters — If safety is "everyone's job" (as xAI suggests) or based on "vibe-based" model cards, it lacks the teeth to stop a truly capable model from being misused.
- In the book — Zvi points out that Anthropic's recent commitments were "never real in an important sense" because they were not technically binding and were easily adjusted when convenient.
The Danger of "Forbidden" Research
- What it is — The pursuit of high-risk interpretability or "intentional design" techniques that could inadvertently reveal "forbidden" capabilities or "destructive weapons" of intelligence.
- Why it matters — Even if a model is currently small, researching how to "uncover" its internal workings creates a blueprint that can be applied to larger, more dangerous models.
- In the book — Zvi uses the "hidden weapon in the cave" analogy: if you go into a cave to see if a weapon is there, you might just find it and then someone else can steal it.
Frameworks & tools
The "Middle Game" Assessment
Use this to evaluate whether a technology is a "toy" or a "frontier" development.
- Cycle Speed: Is the iteration cycle getting faster?
- Human-out-of-the-loop: Is the AI doing the work of the next iteration?
- Lab Divergence: Are different labs building fundamentally different "services" on themselves?
- Transformational Change: Is the world actually changing, or is it just "Star Trek" (modern life with a different metaphor)?
The "Active Player" Analysis
To determine if a company is a true "Live Player" in the AI race:
- Talent Acquisition: Are they hiring the best safety and capability researchers?
- Compute Access: Do they have the physical infrastructure to train frontier models?
- Product vs. Research: Is the company trying to sell ads (Meta) or build the "Singularity" (Anthropic/OpenAI)?
Memorable examples & stories
The "Star Trek" Metaphor
Zvi describes the "S-curve" believers as people who want the future to look like Star Trek—modern humanity doing modern things, but with "cool intelligence" behind everything. The lesson: People often project their own desires for a "normal" future onto the AI trajectory to avoid the discomfort of true transformation.
The "Hidden Weapon" Cave
Zvi describes a scenario where a researcher finds a "secret destructive weapon" in a cave. They think they are doing it for safety, but they are actually just making the weapon available to the "bad guy." The lesson: There is no "good" reason to perform research that reveals dangerous capabilities if the only reason to do it is to see if they exist.
The "Department of War" (DoW) Conflict
The friction between Anthropic and the government over domestic mass surveillance. Anthropic's refusal to allow their tools to be used for "deanonymizing" and "connecting" all data points. The lesson: The "red line" isn't just about autonomous weapons; it's about the unprecedented power of AI to analyze every piece of data ever collected.
Key quotes
"I think the middle is real... you're seeing cycles get faster and faster but you aren't seeing true transformational changes to the world."
Why it matters: It defines the current era of AI as a period of accelerating internal dynamics before the rest of the world fully changes.
"People desperately want to tell themselves a story... where everything is going to be normal in a fundamental sense."
Why it matters: It identifies the psychological barrier to accepting the reality of recursive self-improvement.
"The problem will still happen. If the model is efficiently advanced, it's efficiently large."
Why it matters: It warns that "small" models don't provide a safety buffer; any breakthrough in a small model can be scaled to a large one.
How to apply it
This week
- Audit your "AI-Normal" bias: Identify one area where you are assuming AI will only be "Internet-big" and research the actual capabilities of current "middle-game" models.
- The "Two-Model" Workflow: Adopt Zvi's personal workflow: Use GPT-5 for "thinking" and Claude Opus for "research/writing" to see if it improves your output.
Ongoing
- Monitor the "Middle Game" signals: Look for signs of humans being removed from the loop in coding or research tasks.
- Practice "Vibe-Check" Skepticism: When a lab releases a "vibe-based" model card, ask: "What are the actual technical constraints here?"
Common pitfalls
- The S-Curve Trap: Assuming that because we haven't hit "Superintelligence" yet, the progress will naturally slow down. Zvi argues that the "middle" is where the acceleration happens.
- The "Everyone's Job" Fallacy: Believing that safety is a collective social responsibility without dedicated, high-talent teams and specific mandates.
- The "Small Model" Safety Illusion: Thinking that testing a dangerous technique on a small model is "safe" because the results won't scale.
Connects to
- The Alignment Problem (Brian Christian): Extends the discussion on how we might actually "align" a system that is rapidly self-improving.
- Superintelligence (Max Tegmark): Challenges the timeline of the "End Game" by placing us firmly in the "Middle Game."
If you remember only three things
- We are in the "Middle Game": AI is beginning to drive its own improvement (Recursive Self-Improvement).
- The "S-curve" is a psychological comfort, not a physical certainty of the AI trajectory.
- Researching "forbidden" capabilities—even on small models—is a violation of "bad virtue ethics" because it creates a blueprint for misuse.
Internalize it
- Reflection questions
- If AI begins to drive its own research, what is the "point of no return" for human oversight?
- In what ways am I currently falling for the "S-curve" trap in my own professional planning?
- How would my career/life change if I accepted that "normal" is no longer a guaranteed outcome of the AI revolution?
- Flashcards
Q: What is the "Middle Game" of AI?
A: The period where AI-driven cycles are accelerating and recursive self-improvement begins, but before humans are completely out of the loop.
Q: What is the "End Game"?
A: The point where AI-driven recursive self-improvement makes human research talent irrelevant.
Q: Why is the "S-curve" a problematic mental model?
A: Because people use it to justify the belief that the future will remain "normal," ignoring the accelerating reality of AI development.
Q: What is the "Hidden Weapon" analogy?
A: The idea that researching "forbidden" capabilities on small models is dangerous because it provides a blueprint for others to use on larger models.
Q: What is Zvi's critique of "vibe-based" safety?
A: That safety measures based on "vibies" or general commitments without technical enforcement are not real commitments.
Fidelity note
Confidence is high. This summary is based strictly on the provided transcript. Note that some specific political details (e.g., the specific names of individuals in the Department of War) are mentioned in the transcript but the focus remains on Zvi's analysis of the implications of those events rather than the politics themselves. The transcript mentions "2026" as a date, which suggests a future-dated or hypothetical context within the podcast's narrative.