Jeremy Howard — End of Finetuning

In one sentence

The concept of "fine-tuning" is a misunderstanding of training dynamics; the correct approach is "continued pre-training," where a model maintains a diverse data mix to prevent catastrophic forgetting while acquiring new capabilities.

Why read this (and who it's for)

This conversation is for AI researchers, engineers, and practitioners who are frustrated by "catastrophic forgetting"—where a model loses its general intelligence while being trained on a specific task. It provides a fundamental shift in how you think about model training: moving away from "fixing" a finished model and toward a holistic, continuous data-curation strategy. You will leave this with a new mental model for training data mixtures and a critique of why current "alignment" and "fine-tuning" methods may actually be degrading model performance.

The big idea

Jeremy Howard argues that the industry has adopted a "three-step" approach to NLP (pre-training, then fine-tuning) that is fundamentally flawed because it treats the model as a static object. Because the model forgets general skills when exposed to a narrow, high-intensity data stream (like pure code or pure instructions), the "fine-tuned" result is often a degraded version of the original.

The solution is to abandon the idea of a "fine-tuning" phase entirely. Instead, we should treat all training as continued pre-training. In this model, you never "throw away" the general data. Instead, you maintain a diverse mixture of data—including instructions, code, and general document completion—and gradually curate that mixture to become higher quality and more specific as the model matures. By keeping the "general" data in the mix, you ensure the model retains its foundational capabilities while it learns new ones.

Key ideas

The Fallacy of Fine-Tuning

Fine-tuning is not a separate "mode" of learning; it is simply pre-training on a different distribution. Howard argues that because the "fine-tuning" stage often lacks the diversity of the original pre-training, it causes the model to "forget" how to do things it previously knew. Why it matters: If you believe fine-tuning is a magic "add-on," you will continue to be surprised by why your models lose their general reasoning abilities when trained on specific tasks. In the book: Howard points to CodeLlama as a primary example. While it became excellent at coding, it became worse at general tasks because Meta only included 0.3% non-code data during its 500B token "fine-tuning" phase.

Catastrophic Forgetting and the "Alignment Tax"

When a model is trained on a narrow dataset (like RLHF or specific task instructions), it often loses the ability to perform general tasks. This is "catastrophic forgetting." Howard suggests that many of the issues people see as "alignment taxes" are actually just the result of poor training practices where the model is being forced to forget its general knowledge. Why it matters: It shifts the blame from "inherent limitations of LLMs" to "poor training methodology." In the book: Howard notes that the "less aligned" versions of models often perform better on general tasks because they haven't been subjected to the "forgetting" caused by narrow alignment training.

Continued Pre-training as a Unified Strategy

Instead of a linear pipeline (Pre-train $\rightarrow$ Fine-tune $\rightarrow$ Align), Howard proposes a continuous loop. You start with a broad data mix and, as you progress, you curate that mix to be higher quality and more specific, but you never stop including the general data. Why it matters: This preserves the "latent" capabilities of the model while allowing it to specialize. In the book: Howard argues that the "right way" to do this is to include all types of data (instructions, exercises, code, etc.) from the start and gradually refine the quality of that data.

The Importance of Data Diversity

The reason models fail during "fine-tuning" is a lack of data diversity. If you only show a model code, it forgets how to be a general assistant. Why it matters: It highlights that the composition of the data is as important as the amount of data. In the book: Howard emphasizes that the "right" way to train is to include "all the kinds of data that you care about" from the very beginning.

The "Clunk" in Training Curves

Howard observes a strange phenomenon where training loss "clunks" or drops suddenly at the end of every epoch. While some claim this is "normal," Howard suspects it indicates a fundamental issue with how training is being managed or how data is being handled. Why it matters: It suggests that our current tools (like the Hugging Face Trainer) might be hiding underlying issues in training dynamics. In the book: Howard notes that while others say "don't worry about it," he believes we need to understand these dynamics to truly master model training.

Frameworks & tools

The Continued Pre-training Framework

Instead of the standard 3-step fine-tuning process, use this continuous approach:

1. Initial Mixture: Start with a massive, diverse dataset including general-purpose document completion, code, and instructions.
2. Continuous Inclusion: Never remove the "general" data from the training loop.
3. Gradual Curation: As training progresses, increase the weight/quality of the specific tasks you want the model to master (e.g., higher-quality reasoning or specific coding tasks).
4. Validation: Monitor for "forgetting" by testing on general benchmarks throughout the entire process, not just at the end.

Memorable examples & stories

The CodeLlama "Forgetting" Case

Meta released CodeLlama, which was "fine-tuned" on 500 billion tokens of code. While it became a coding powerhouse, it became significantly worse at general tasks. The Lesson: This happened because they only included 0.3% non-code data. It proves that "fine-tuning" without diversity causes catastrophic forgetting.

The "Chinese Room" Inspiration

Howard explains how his interest in AI started with the "Chinese Room" thought experiment—the idea of a system that can manipulate symbols perfectly without "understanding" them. The Lesson: This led him to believe that if a neural net could ingest enough text to mimic these patterns, it would functionally be intelligent, regardless of the internal "consciousness."

The "Clunk" Mystery

Howard describes a strange behavior where training curves "clunk" at the end of every epoch. When he asked the community, everyone said "it's normal," but he refuses to accept that because it's a new phenomenon in 30 years of neural network history. The Lesson: Don't accept "it's just how it is" as an answer for training dynamics; there is likely a technical reason we haven't identified yet.

Key quotes

"There's no such thing [as fine-tuning]. There's only continued pre-training." Why it matters: It challenges the fundamental vocabulary of the current AI industry.

"I've always been somebody who does not want to build stuff on lots of big computers because most people don't have lots of big computers and I hate creating stuff that most people can't use." Why it matters: This explains the philosophy behind Fast.ai—democratizing access to high-level research.

"The right way to do this is to fine-tune language models, is to actually throw away the idea of fine-tuning." Why it matters: It's a paradoxical way of saying we need to stop treating the process as a separate, isolated step.

How to apply it

This week

• Audit your training data: If you are training a model on a specific task (e.g., SQL generation), look at your data mix. What percentage is "non-task" data?
• Identify "Forgetting": Run a benchmark on a general-purpose task (like basic reasoning) before and after you run a training run on a specific task. See if the "fine-tuning" is actually hurting the model's general abilities.

Ongoing

• Adopt the "Continued Pre-training" Mindset: Whenever you plan a training run, ask: "How can I keep the general data in this mix while still improving the specific task?"
• Prioritize Data Quality over Quantity: Instead of just adding more tokens of a specific task, focus on "gradually curating" higher-quality data within the mixed training set.

Common pitfalls

• The "Standard" Trap: Just because the 3-step approach (Pre-train $\rightarrow$ Fine-tune $\rightarrow$ Align) is what everyone does (including OpenAI), doesn't mean it's the optimal way to train a model.
• Over-reliance on RAG: Howard warns that RAG is often an "inefficient hack." While it's useful, it doesn't incorporate information into the weights of the model as efficiently as a properly executed fine-tuning (or continued pre-training) would.
• Ignoring Training Dynamics: Many people focus on the output of the model without understanding the dynamics of how the model is learning (the "loss surface" and "training curves").

Connects to

• Retrieval Augmented Generation (RAG): Howard argues RAG is a "hack" compared to the nuanced integration of information into model weights via fine-tuning.
• The Chinese Room Argument: The philosophical foundation for why large-scale text ingestion can lead to functional intelligence.
• Catastrophic Forgetting: The technical hurdle that makes "fine-tuning" difficult to execute without destroying the model's prior knowledge.

If you remember only three things

• Fine-tuning is a myth: It is actually "continued pre-training" where you must maintain a diverse data mix to avoid "catastrophic forgetting."
• Diversity is the cure: To keep a model smart while teaching it new things, you must never stop showing it general-purpose data (code, instructions, etc.).
• The "Alignment Tax" is often a "Training Error": Many of the problems we see in model behavior are caused by training models on too-narrow data, which causes them to forget their general capabilities.

Internalize it

• Reflection questions

  • If I were to train a model to be a better coder today, what "general" data would I keep in the mix to ensure it doesn't lose its ability to write poetry or explain logic?
  • Why am I currently treating "fine-tuning" as a separate step? What would happen if I viewed it as a continuous process instead?
  • How can I apply the "gradual curation" principle to my own learning? How can I maintain a broad "general" knowledge base while specializing in a new skill?

• Flashcards Q: What is the difference between "fine-tuning" and "continued pre-training" according to Jeremy Howard? A: There is no difference; fine-tuning is simply pre-training on a different data distribution. The goal is to treat it as a continuous process.

Q: What is "catastrophic forgetting" in the context of LLMs? A: It is when a model loses previously learned skills (general knowledge) because it is being trained on a narrow, task-specific dataset.

Q: Why did Howard cite CodeLlama as an example of "bad" fine-tuning? A: Because it was trained on 500B tokens of code with only 0.3% non-code data, causing it to lose its general-purpose capabilities.

Q: What is the "Continued Pre-training" strategy? A: Maintaining a diverse data mix (instructions, code, general docs) from the start and gradually curating higher-quality data while never removing the general data.

Q: How does Howard view RAG (Retrieval Augmented Generation)? A: He views it as an "inefficient hack" compared to the nuanced way information is incorporated into model weights during fine-tuning.

Fidelity note

This summary is grounded in the provided transcript. I am confident in the distinction between "fine-tuning" and "continued pre-training" as it is the central thesis of the conversation. The transcript mentions "CodeLlama" and "RLHF" as specific examples of catastrophic forgetting. The transcript does not provide a specific mathematical formula for the "gradual curation" process, but describes it as a qualitative shift toward higher-quality, more specific data while maintaining the general data mix.