Simon Willison — No Moat: Closed AI gets its Open Source wakeup call

In one sentence

The "moat" of being a large, closed-source AI company is evaporating because the open-source community can iterate on smaller, stackable models at a velocity that exceeds the bureaucratic and structural constraints of giants like Google and OpenAI.

Why read this (and who it's for)

This conversation deconstructs the "closed vs. open" debate by looking at the internal panic within Google. It is for AI researchers, developers, and product leaders who need to understand why "bigger" is no longer a winning strategy and how to identify real competitive advantages—specifically in data loops and architectural efficiency—as the underlying models become commoditized.

The big idea

The central thesis, derived from a leaked internal Google memo, is that Google and OpenAI have "no moat" against the "mighty open source horde." While the giants have focused on the sheer scale of training massive models, the open-source community has achieved comparable capabilities in a fraction of the time. The sheer speed of innovation in the open-source ecosystem creates a "race to the bottom" for base model capabilities, eventually turning high-quality LLM intelligence into a commodity.

Because the base models will eventually become commoditized, the real competitive advantage shifts to three areas: specialized, optimized models (running on-device or in constrained environments), proprietary first-party data loops (the "sand" of the AI world), and stackable innovations like LoRA, which allow for rapid, modular improvements without the need to retrain massive weights from scratch.

Key ideas

The Speed of Open Source

Open source is not just a philosophical choice; it is a structural advantage for innovation. The memo highlights that it took the open-source community only a couple of months to reach the level of capability seen in early closed models. Why it matters: If you are a closed-source company, your "lead" is temporary and easily eroded by a global, decentralized research community that can iterate without the friction of corporate hierarchy. In the book: Simon notes that while some models aren't yet at GPT-4 standards, they are "within spitting distance," and the speed of this progress is what should alarm the incumbents.

The Fallacy of "Size as a Moat"

The memo argues that large models are not necessarily a competitive advantage. In fact, smaller models that can be fine-tuned on a laptop in a couple of hours are often more useful for real-world deployment. Why it matters: If the "big" model is the commodity, the winner is whoever can make the "small" model most useful for a specific task. In the book: The memo suggests that having to retrain a giant model every few months is less useful than having a small model that can be rapidly iterated upon by the community.

Stackable LoRA (Low-Rank Adaptation)

LoRA is a technique where you freeze the main model and only train a small, "stackable" set of weights. This allows for modular improvements (like adding a specific skill or style) without retraining the whole system. Why it matters: This creates a "package" economy. A base model can be 5 years old, but if it has a thousand "stacks" of LoRAs trained by the community, it can outperform a brand-new model. In the book: Simon compares this to the software ecosystem: just as a new programming language is useless without a library of packages, a new model is less useful than an old one with a massive ecosystem of stackable improvements.

First-Party Data Loops

As models become commoditized, the "moat" shifts to the data used to train and refine them. The winner will be the one who owns the most valuable, high-quality "first-party" data. Why it matters: Anyone can download a model from Hugging Face, but not everyone can access the unique, high-quality data generated by millions of users interacting with a specific product (like Google Docs or Sheets). In the book: Alessio Fanelli points out that the "sand" of the AI world is the data. The question for a startup is: "How do I get enough people to generate data for me so that I can still make it good?"

On-Device and Specialized Optimization

There is a massive, untapped market for models that run on-device (phones, laptops, even fighter jets) where latency and privacy are non-negotiable. Why it matters: While many people want "chat" bots, there are "life or death" situations where you cannot call an API. In the book: Travis Fischer mentions the need for "reliable primitives" and high-performance optimization (like Mojo) to run models in environments where milliseconds matter.

Frameworks & tools

The "Commodity" Analysis Framework

To determine where your actual competitive advantage lies in the AI era, evaluate your position across these three tiers:

1. The Model Layer (Commodity): Is your advantage based on a "better" model? (Warning: This is a race to the bottom).
2. The Data Layer (The Moat): Do you have a "first-party loop" where users generate unique, high-quality data that others cannot access?
3. The Application Layer (The Value): Can you build a "reliable primitive" or a specialized tool (e.g., for a fighter jet or a specific enterprise workflow) that solves a problem a general-purpose chatbot cannot?

The "Stackable" Development Strategy

Instead of trying to build the "ultimate" model, focus on:

1. Base Selection: Choose a high-performing open-source base model (like Llama).
2. Modular Training: Use LoRA to train specific, stackable skills.
3. Community/Internal Iteration: Rapidly iterate on these "stacks" to create a specialized tool without the overhead of full-model retraining.

Memorable examples & stories

The "Llama" Effect

Meta released Llama, and the community immediately built a "tree" of models (Alpaca, Vicuna, etc.) on top of it. The lesson: Open source creates a "free" labor force of researchers and developers who optimize the model for you.

The "Romance Scam" Warning

Simon discusses how a model running on a laptop could be used to automate romance scams at a scale never before possible. The lesson: The risk of AI isn't just "sci-fi" (AI killing humans); it's the "non-fiction" risk of bad actors using cheap, accessible models to scale existing harms.

The "Mojo" Performance Jump

Jeremy Howard (Fast.ai) showed that by using Mojo to implement matrix multiplication, he got 2,000x the performance of Python. The lesson: Language supersets (like Mojo being a superset of Python) allow for "incremental adoption"—keeping the familiar interface while unlocking massive performance gains.

Key quotes

"We have no moat, and neither does OpenAI." Why it matters: This is the "mic drop" of the Google memo, acknowledging that the gap between closed and open models is closing at a terrifying speed.

"The marginal price of intelligence will go to zero." (Attributed to Sam Altman) Why it matters: If intelligence becomes a commodity, you must find value elsewhere (data, UX, or specialized hardware).

"Secrecy is a moot point... we are already sharing everything with them in the form of the steady flow of poached senior researchers." Why it matters: You cannot hide your "secret sauce" if your best people are constantly moving to the competition.

How to apply it

This week

• Audit your "Moat": Identify which part of your product relies on "model intelligence." If that intelligence is becoming a commodity, what is the other part of your product that is hard to replicate?
• Explore Local LLMs: Download a model like Vicuna and run it on your local hardware. Understand the constraints and capabilities of "on-device" AI.
• Identify your "First-Party Loop": Map out exactly what data your users generate when using your product. Is that data high-quality? Could it be used to train a specialized model?

Ongoing

• Prioritize Data over Weights: Shift your engineering focus from "how do we make the model smarter" to "how do we get better data from our users."
• Adopt a "Stackable" Mindset: When building features, think about how they can be modularized (like LoRAs) rather than being hard-coded into a monolithic system.
• Monitor the "Open Source Horde": Regularly check Hugging Face and the Llama ecosystem to see what the community is building; use their "free" innovation to accelerate your own roadmap.

Common pitfalls

• The "Big Model" Trap: Thinking that having a larger, more capable model is a sustainable competitive advantage. In a commoditized market, the "biggest" model is often the hardest to deploy and the slowest to update.
• Ignoring the Data Moat: Focusing so much on the "AI" part of the business that you forget the "business" part—which is the unique data and user behavior that only your company can capture.
• Underestimating Open Source Speed: Assuming that "closed" models will always be 6 months ahead. The memo proves that the gap can be closed in weeks by a global community of developers.

Connects to

• "The Age of AI" by Henry Kissinger/Eric Schmidt: Provides the geopolitical and historical context for why these "moats" are so hard to build.
• "Prediction Machines" by Agrawal, et al.: Explores the economics of AI as a way to lower the cost of prediction, which aligns with the "commoditization of intelligence" argument.

If you remember only three things

• Intelligence is becoming a commodity: The "marginal price of intelligence" is heading toward zero, meaning the model itself is not your long-term moat.
• Open source is a force multiplier: The speed of the open-source community allows for a level of iteration and "stackable" innovation (LoRA) that closed systems struggle to match.
• The Moat is in the Data: Your true competitive advantage lies in your "first-party data loops"—the unique, high-quality data generated by your specific users.

Internalize it

• Reflection questions
  • If a competitor could give you a model that is 99% as capable as yours for free, what would remain of your business?
  • What is the "first-party data" my company produces that no one else can access?
  • How can I move from "using a general-purpose AI" to "owning a specialized, optimized tool" for my specific niche?
• Flashcards Q: What is the core argument of the "No Moat" memo? A: That Google and OpenAI have no significant advantage over the open-source community because open source can iterate faster and develop "stackable" improvements (LoRAs) more efficiently. Q: Why is "size" no longer a guaranteed moat? A: Because smaller, stackable models can be fine-tuned quickly on a laptop, making them more practical for many real-world applications than massive, slow-to-update models. Q: What is a "first-party data loop"? A: A system where users interact with a product to generate high-quality, unique data that can be used to further train and refine the model. Q: What are the benefits of LoRA in the context of LLMs? A: It allows for "stackable" improvements—training small sets of weights to add skills or styles without needing to retrain the entire base model. Q: What is the "commoditization of intelligence"? A: The idea that as models improve, the cost of generating a high-quality response will drop toward zero, making the underlying model a commodity.

Fidelity note

This summary is grounded in the provided transcript of the conversation between Simon Willison, Travis Fischer, and the host. I am confident in the technical descriptions of LoRA and the "No Moat" argument. Note that the "No Moat" memo itself is a leaked internal document; while the transcript treats it as a credible source of internal panic, the actual contents of the memo were not read aloud in full, but rather summarized by Simon Willison and the participants.