Sebastian Raschka — State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI

In one sentence

The competitive advantage in AI is shifting from "secret" research breakthroughs to the ability to mobilize massive compute, optimize data pipelines, and maintain a high-velocity research culture.

Why read this (and who it's for)

This conversation provides a high-level strategic map of the AI industry's current trajectory. It is for researchers, engineers, and investors who want to understand the "meta-game" of AI: why certain companies are winning, the reality of the "DeepSeek moment," and the technical bottlenecks (like tool-use and robotics safety) that will define the next 12–24 months. You will be able to identify which parts of the AI stack are becoming commoditized and where the real "moats" are being built.

The big idea

The "DeepSeek moment" (January 2025) signaled a shift where high-performing, open-weight models could be produced with significantly less compute than previously thought. This has democratized access to state-of-the-art (SOTA) weights, meaning that "winning" is no longer about who owns a secret formula, but who can execute on that formula with the most resources.

The competition is now a race of scale, culture, and data. While the "ideas" in AI are fluid and move quickly between labs as researchers change jobs, the winners are determined by who can build the best infrastructure (like Google’s TPUs), curate the highest-quality data (the "secret" sauce of frontier labs), and maintain a culture of rapid iteration. We are moving toward a world where the "best" model is simply the most recent one released, and the differentiator is the sheer volume of compute and human effort poured into the training run.

Key ideas

The "DeepSeek Moment" and Open-Weight Dominance

DeepSeek proved that near-SOTA performance could be achieved with much lower compute costs. This sparked a massive movement in China, where multiple companies (Minimax, Z.ai, Kimi) are now releasing strong open-weight models.

• Why it matters: It creates a "flywheel" of influence. Because many global companies won't pay for Chinese APIs due to security concerns, open-weight models provide a way for Chinese labs to capture the global market share of AI expenditure.
• In the book: The guests note that while DeepSeek might be "losing its crown" as the sole preeminent maker, it paved the way for a massive ecosystem of Chinese competitors.

The "Culture" of Innovation

The difference between companies like Anthropic, OpenAI, and Google is often their organizational culture. Anthropic is noted for its focus on "less chaotic" execution and a heavy bet on coding (Claude Code).

• Why it matters: When ideas flow freely between labs, the winner is the one who can actually land the product. OpenAI's "chaotic" startup culture has proven highly effective at launching "definitional" products like Sora and o1.
• In the book: The guests argue that while Google has structural advantages (TPUs, scale), OpenAI’s ability to manifest a new research idea into a product is a core organizational trait.

Tool Use as a Paradigm Shift

A major unlock in the next year is "Tool Use"—the ability for an LLM to autonomously call a web search, a Python interpreter, or a calculator to solve a problem.

• Why it matters: This solves the "hallucination" problem. Instead of asking a model to remember the 1998 World Cup winner, the model realizes it needs a tool, performs a search, and reports the fact.
• In the book: The guests highlight that while this is a massive unlock, it requires a "trust" layer (containerization) because users are wary of giving models access to their local systems.

The Data Moat

As algorithmic innovations become more common, the "secret" of frontier labs is the data. The winner is the one who finds the best, cleanest, and most unique data (e.g., high-quality scientific PDFs or specific Reddit threads).

• Why it matters: When everyone has access to the same "standard" internet scrape, the only way to get a better model is to find data others haven't used.
• In the book: The guests mention that "the sexiest thought of a scientist" is often just "I'm going to make the data better" or "I'm going to make the infrastructure 5% faster."

The "Reasoning" vs. "Style" Split

There is a distinction between RLHF (Reinforcement Learning from Human Feedback) and RLVR (Reinforcement Learning from Verifiable Rewards).

• Why it matters: RLHF often just teaches a model a "style" (the average preferred answer). RLVR allows a model to solve complex, difficult problems by providing a verifiable signal of success.
• In the book: The guests suggest that for reasoning-heavy tasks, we should allocate more budget to RLVR because it scales with complexity.

Frameworks & tools

The "Research-to-Product" Moat

To understand why a company is winning, evaluate them across these three dimensions:

1. Hardware/Infrastructure: Does the company own the silicon or the data center? (e.g., Google's TPU advantage).
2. Cultural Velocity: Can the lab turn a research paper into a product in weeks? (e.g., OpenAI's "chaotic" but effective landing).
3. Data Superiority: Does the lab have access to "closed-garden" data or a unique way of cleaning/processing public data?

The "Tool-Use" Architecture

When building agents, move away from "all-in-one" prompts toward a modular system:

1. Planner: The LLM decides which sub-tasks are needed.
2. Executor: The LLM calls a specific tool (Search, Python, Calculator) for each sub-task.
3. Aggregator: The LLM stitches the results back together into a final answer.

Memorable examples & stories

The "Debug" Drink of Water

One guest describes the feeling of debugging as "a drink of water after you've been going through a desert for days."

• The Lesson: AI is not just about generating new content; it is about removing the "desert" of mundane, frustrating troubleshooting from the human experience.

The "Common Crawl" vs. Licensed Data

The discussion on how "Common Crawl" (a scrape of the whole internet) is largely unlicensed means that while people are happy to have their sites used, they haven't explicitly consented to the specific way the data is used for training.

• The Lesson: The "legal" layer of AI is a mess of "implied" permissions that will be the battleground for data ownership.

The "Junior vs. Senior" Coding Paradox

Data shows that senior developers are shipping more AI-generated code than juniors.

• The Lesson: Experts are better at reviewing and trusting the output of an imperfect model. The challenge for the future is how juniors will learn the "fundamentals" if they never struggle with the "doing" themselves.

Key quotes

"I don't think nowadays... there will be any company that has access to technology that no other company has access to... The differentiating factor will be budget and hardware constraints." — Sebastian Raschka

• Why it matters: It shifts the focus from "magic" to "resources."

"The most common complaints about LLMs are... hallucinations... one of the best ways to solve hallucinations is to not try to always remember information... why not use a calculator app?" — Nathan Lambert

• Why it matters: It defines the shift from "Knowledge Models" to "Agentic Tools."

"If you're doing a PhD... the life afforded to people to work at these AI companies... is transformative for your life." — Sebastian Raschka

• Why it matters: It highlights the massive economic reality of the AI "gold rush."

How to apply it

This week

• Audit your "Tool Use": Identify one repetitive task you do (e.g., "summarize this long PDF") and see if you can break it into a "Tool" call (e.g., "Extract key dates, then summarize").
• Identify your "Data Moat": If you are building a project, ask: "What data do I have access to that a general model (like GPT-4) cannot see?"

Ongoing

• Prioritize RLVR over RLHF: When training or fine-tuning, look for "verifiable" rewards (like a code test passing) rather than just "human-preferred" labels.
• Monitor the "DeepSeek" Effect: Watch for Chinese open-weight models to see what "free" capabilities are becoming available to you as a developer.

Common pitfalls

• The "Algorithm" Fallacy: Many believe there is a secret "God Model" algorithm. The experts argue that the "secret" is often just better data, more GPUs, and a faster-moving team.
• The "Human-in-the-loop" Trap: People assume robots will be in homes soon. The experts warn that the "safety" and "continual learning" requirements for a robot in a human home are orders of magnitude harder than a chatbot.

Connects to

• The Scaling Laws: This conversation expands on the "Scaling Laws" by focusing on the financial and logistical costs of scaling (serving vs. training).
• Agentic Workflows: This connects to the idea of "Agentic" behavior—moving from a chatbot to a system that uses tools.

If you remember only three things

• The DeepSeek Moment: Open-weight models from China are making SOTA performance accessible and cheaper.
• Tool Use is the Unlock: The next big leap isn't "smarter" models, but models that can use a calculator, a browser, and a Python interpreter to stop hallucinating.
• Data is the Moat: In a world of infinite compute, the winner is the one with the best, most unique data.

Internalize it

• Reflection questions

  • If "ideas" are becoming a commodity, what is my unique "data" or "resource" advantage?
  • How can I incorporate "Tool Use" (calculators, search) into my current workflow to eliminate hallucinations?
  • If the "junior" developer learns by doing, how can I ensure I am still "doing" the hard work while using AI to accelerate the boring parts?

• Flashcards Q: What was the "DeepSeek Moment"? A: The release of high-performing open-weight models with significantly lower compute costs, sparking a massive movement in Chinese AI labs.

Q: Why is "Tool Use" considered a paradigm shift? A: It allows models to solve hallucinations by calling external tools (like a calculator or web search) instead of trying to memorize facts.

Q: What is the difference between RLHF and RLVR? A: RLHF trains on "preferred" human labels (style), while RLVR trains on "verifiable" rewards (solving complex problems).

Q: What is the "Data Moat" in the current AI landscape? A: Because everyone has access to the same internet scrapes, the winning labs are those that find, clean, and use unique, high-quality data.

Q: Why is the "Junior vs. Senior" coding gap important? A: It shows that experts are better at reviewing and trusting AI-generated code; it raises the question of how juniors will learn the fundamentals.

Fidelity note

This summary is grounded in the provided transcript. I am confident in the technical distinctions made between RLHF/RLVR and the "Tool Use" definitions. The discussion on "DeepSeek" and Chinese-led open-weight models is a central theme of the conversation. Note that the "DeepSeek moment" refers specifically to the period around January 2025.