6 interviews
Nathan Lambert is an AI researcher and thinker who focuses on the technical and strategic foundations of Large Language Models (LLMs). He explores the engineering of RLHF, the competitive landscape of the AI industry, and the practical evaluation of open-source models.
Llama 2 serves as a commercial catalyst for the open-source ecosystem by providing a high-quality foundation for domain-specific fine-tuning. It is evaluated on its ability to handle complex instructions and its suitability for RAG or tool use.
From Llama 2: The New Open LLM SOTA (ft. Nathan Lambert, Matt Bornstein, Anton Troynikov, Russell Kaplan, Whole Mars Catalog et al.) (episode)
Meta invested heavily in preference data to train two separate reward models for helpfulness and safety. This represents a move beyond simple next-token prediction toward aligning model behavior with human preferences.
From Llama 2: The New Open LLM SOTA (ft. Nathan Lambert, Matt Bornstein, Anton Troynikov, Russell Kaplan, Whole Mars Catalog et al.) (episode)
RLHF is not about teaching a model new facts or moral values, but rather about shaping the probability distribution of the model's responses. It treats 'good' behavior as a maximum utility function modeled via pairwise comparisons.
From RLHF 201 - with Nathan Lambert of AI2 and Interconnects (episode)
RLHF relies on the mathematical assumption that human preferences are measurable. Specifically, the Bradley-Terry model allows for predicting the probability that a human will prefer one response over another.
From RLHF 201 - with Nathan Lambert of AI2 and Interconnects (episode)
A standard RLHF pipeline requires a base model that is instruction-tuned and a reward model that assigns a scalar score to a completion. This infrastructure is used to align model outputs with human preferences.
From RLHF 201 - with Nathan Lambert of AI2 and Interconnects (episode)
Every idea names the summary it came from. Written by a local model from this mind’s own work, never from memory.