What Training Actually Changes: Jeremy Berman on Building Language Models and Teaching Them to Reason

1. Guest Background

Jeremy Berman, the guest in this approximately 56-minute conversation with Matthew Berman, is introduced as an Anthropic researcher and a self-taught AI researcher. Those are the supported biographical anchors. The episode does not establish a detailed employment history, identify his contributions to particular products, or document specific research achievements. His institutional role provides context for the discussion, while the substance comes from the explanations and demonstrations he offers.

The conversation asks how language models are built: what neural networks represent, how training adjusts them, why text prediction produces a base model, and what additional work creates assistant behavior and reasoning capabilities. It also considers whether extended reasoning could support new discoveries. Jeremy explicitly aims for an intuitive overview, acknowledging that individual subjects could sustain much deeper treatment.

His teaching approach makes that scope tangible. An interactive apartment-pricing example turns parameters into adjustable controls; a short PyTorch demonstration exposes the training loop; an open-weight model shows the behavioral differences between completion and instruction following. These are relevant demonstrations of technical understanding, rather than disclosures of Anthropic’s internal training procedures. Jeremy also encourages learners to pursue the questions that capture their interest. His self-taught background gives that advice personal context, but the episode supplies neither a complete account of his own learning journey nor a guaranteed route into research.

2. What the Episode Covers

Jeremy begins with a supervised learning problem that has a visible target. A table of New York apartments contains location, square footage, bathroom count, and actual sale price. The task is to learn a function connecting the first three to the last. Matthew considers how manual rules would cope with thousands or millions of examples; Jeremy adds the complication of moving from three inputs to a thousand. Learning parameters becomes a way to avoid specifying every relationship by hand.

His first model multiplies the inputs by adjustable values and adds the results. The illustrative initial weights are 0.25 for size, 0.05 for bathrooms, and 0.15 for location. These are randomized teaching values, not estimates of housing-market relationships. Calling them knobs makes the effect of adjustment visible. Jeremy then qualifies the analogy: location and size can interact, and a more complex function may be needed. Layers feed one set of computations into another, creating nested functions rather than a single weighted sum.

Training automates the adjustment. A loss measures disagreement between predictions and labels. Automatic differentiation tracks the computations involving each parameter and calculates how the loss changes locally with that parameter. Backpropagation uses the chain rule; the learning rate controls the size of the update. The direction therefore comes from calculation rather than random experimentation with every control. His PyTorch example runs predictions, calculates loss, clears gradients, backpropagates, and updates parameters. It uses mean squared error over 1,000 training steps, printing every 100 steps. Jeremy explicitly calls it a toy demonstration rather than a production recipe.

The same apartment records then become a bridge to language modeling. Written as text, they can be learned as sequences: a location is followed by punctuation, an area, and eventually a price. The actual units are tokens, which need not be whole words. Instead of producing only a numerical price, the model assigns probabilities to possible next tokens. Training increases the probability of the observed continuation. Jeremy’s example of assigning only 10 percent probability to “1.2 million” illustrates that objective; the percentage is not a reported performance measurement.

A roughly 500-million-parameter base model demonstrates what this objective does and does not deliver. Its continuations include code, Chinese text, and an apparent invented quotation attributed to Matthew. Jeremy explains that learning to continue text does not itself guarantee truthfulness, helpfulness, or verbatim recall of someone’s actual words.

Assistant training introduces a more specific behavioral target. Jeremy imagines 100,000 examples of user questions and desirable assistant responses in a consistent format, including a stop token. That number explains the mechanism rather than disclosing an actual dataset size. The model learns to emit the marker, and software ends generation when it appears. Chat-interface messages are also converted into the format the model receives. Jeremy recommends inspecting this conversion with the tokenizer, connecting the visible interface to the sequence actually processed.

3. Core Views: Reasoning, Examples, and Limits

The interview’s strongest organizing idea is that several questions often bundled together as model capability should be assessed separately. A model has numerical parameters and a structure describing how those numbers participate in computation. Jeremy’s account of a parameter file plus a program makes parameter count an incomplete description: counting values does not reveal their wiring. Likewise, interpreting a weight as an input’s importance works only within the simple illustration. Once apartment size and location interact, an individual parameter no longer offers that straightforward explanation.

Expressive capacity also differs from successful learning. Jeremy reports that the linear demonstration appears to level off at an error of about $70,000, which he attributes to its simple function. He introduces activation functions and richer networks to explain how more complicated relationships can be represented. That is a useful diagnosis within the example, but his function-approximation discussion does not demonstrate that optimization will find the desired parameters or that predictions will generalize to unseen apartments. The reported error belongs to the demonstration, not to neural-network property valuation in general.

The choice between a specialized network and a language model brings another condition into view: resources. Jeremy favors a dedicated numerical network when the task is simply predicting a price under finite constraints. Language introduces more degrees of freedom and complicated relationships, which he argues require additional data, parameters, and computation. His suggestion that the approaches could converge toward the same mapping in an ideal limit is conditional, and he acknowledges nuance. The practical lesson concerns matching representation to requirements; it does not establish that one architecture wins every real comparison.

A particularly clear distinction separates plausible completion, assistant behavior, and correctness. With user and assistant prefixes, the base model answers one plus one and then continues into incoherent material. The prefix can induce a role without producing dependable conversational conduct. Later, an assistant that follows the format and stops still fails the expression 398 × 923 + 2901. Jeremy describes the outputs approximately, so his spoken numbers should not be treated as exact arithmetic records. Asking for stepwise reasoning can change the response, but he explicitly says it need not produce the right answer. Visible intermediate steps therefore provide something to inspect, not proof of correctness.

Jeremy’s explanation of reinforcement learning addresses the difference between imitating a finished product and learning through attempts. He argues that internet material often preserves the resulting blog post or code, rather than the complete process that produced it. His rejection-sampling example generates a hundred responses to one problem and trains on successful ones. The hundred is illustrative. Keeping successes can work, he says, but discards negative examples and reinforces incidental tokens inside successful responses alongside useful reasoning.

His account of GRPO adds comparison within a group. Responses receive scores for format and correctness; their relative performance supplies an advantage signal used to make stronger responses more likely and weaker ones less likely. This explains why failures can become informative rather than simply being removed. It remains an intuitive account, however, without the complete objective or implementation. A response-level score should not be mistaken for evidence that every reasoning step has received precise credit. Matthew and Jeremy also distinguish this parameter-updating post-training process from the reasoning generated during a user interaction. Prompting and training operate at different points in the system.

The promise of on-policy training is similarly qualified. Jeremy argues that a model already capable of generating its own candidate answer may need gentler adjustments to favor that answer than to imitate somebody else’s reasoning. He nevertheless acknowledges degradation in other capabilities. His example of an assistant adopting the style of newly supplied raw code illustrates the broader tension between learning new material and preserving prior behavior. He emphasizes environments where attempts can be repeated and answers verified, while acknowledging research exploring the merger of training stages.

Finally, the episode offers two claims whose status should remain explicit. Matthew’s recollection that o1 coded better and hallucinated less is an experience judgment without a quantified comparison here. Jeremy’s analogy with Einstein developing abstractions for general relativity supports his prediction that long-horizon reasoning could enable new knowledge. The analogy proposes a mechanism and a research direction; the interview does not demonstrate the predicted discovery capability.

4. Learning and Application

A useful learning exercise is to begin with a narrowly defined supervised problem and explain each component before expanding it. Following the apartment example, specify the inputs, identify the observed labels, and decide whether the required output is a number or text. Then locate the model structure, parameters, loss, and update rule in the training loop. Jeremy’s thousand-step demonstration helps reveal that loop; it does not establish an appropriate training duration for another task. If improvement stalls, ask whether representation is limiting the fit, while remembering that a local gradient cannot certify a final optimum or performance on unseen examples.

For model selection, turn the episode’s resource argument into a comparison question. If the requirement is only a numerical estimate, a specialized network is a reasonable alternative to evaluate. If handling open text is essential, assess what that flexibility contributes and what resources it consumes. This is an application derived from Jeremy’s explanation, not a business outcome demonstrated in the episode. His ideal-limit convergence argument cannot establish equivalence under a particular budget or dataset.

For chat-system diagnosis, inspect formatting separately from answer quality. When a model keeps continuing, examine the tokenizer’s rendering of system and user messages, the assistant prefix, and the stopping mechanism. Correcting a mismatch can address a conversational failure without establishing mathematical competence. Jeremy says his demonstration uses a free Colab T4 and downloadable open weights, making his particular experiment accessible. That historical setup is not a promise that every training exercise is free or that the same resources will always be available.

For everyday prompting, treat a request for stepwise work as a change to the current generation context, then verify the result independently. Do not reuse approximate spoken arithmetic as an answer key. The same discipline applies to quotations: a model sounding like a person does not authenticate the words it attributes to them. Jeremy explicitly warns that exposure to real statements does not guarantee verbatim reproduction. Intermediate reasoning and familiar style can aid inspection, but neither replaces checking the result or its source.

For a feedback-training project, define a verifiable outcome before choosing how to generate and score repeated attempts. The rejection-sampling and GRPO comparison suggests concrete review questions: Are failed responses discarded or used? Are format and correctness being assessed separately? Could rewarding an entire successful response reinforce irrelevant material? These questions help evaluate a proposal, but the interview is insufficient as an implementation specification. Where correctness cannot be assessed reliably, Jeremy’s argument about improvement in verifiable environments cannot simply be assumed to apply.

Finally, evaluate preservation alongside improvement. Following an update, check whether the assistant still behaves as intended and whether other capabilities have deteriorated. On-policy training may soften that tradeoff in Jeremy’s account; it does not remove it. Additional raw code also warrants attention to changes in response style. For discovery-oriented experiments, distinguish producing longer reasoning, proposing an unfamiliar idea, and obtaining an independently confirmed result. That separation allows his argument about emerging abstractions to guide investigation without treating a prediction as an accomplished discovery.

Source

• Original episode: Anthropic Researcher explains how to train LLMs