Recursive Self-Improvement Is Not Just “AI Building AI”: Maker Maker’s Case for Loops, Costs, and Limits

1. Guest Background

This episode comes from a16z / The Ben & Marc Show, hosted by Marc Andreessen and Ben Horowitz. Its title is “Recursive Self-Improvement and the Future of AI,” and a16z uploaded it on 2026-10-12. The episode runs 1,926 seconds, a little over 32 minutes, and it is focused tightly on whether AI systems can improve themselves and what would have to be true for that claim to mean more than ordinary automation.

The identifiable guests are Owen and an unnamed Maker Maker cofounder. The unnamed cofounder introduces himself as someone who studied physics, did a PhD in reinforcement learning at the MIT Media Lab, likes building things, and is now building Maker Maker with Owen and Grace. Owen says he first studied math, became interested in AI through formal language, logic, and theorem proving, worked on continual learning and meta learning during his PhD, later spent several years at DeepMind as a research scientist, and is now working on RSI.

That background matters because the discussion is not a generic futurist panel. The guests are AI researchers and startup builders working on recursive self-improvement at Maker Maker. The supported description of their work includes multi-agent systems and optimization across prompts, harnesses, weights, and chips. The hosts press them on definitions, measurement, cost, and limits; the guests answer from the standpoint of a company trying to turn RSI from a slogan into an engineered system.

2. What the Episode Covers

The episode begins by clearing up what recursive self-improvement is supposed to mean. The guests say RSI is now an overloaded term. Many pitches call themselves RSI because they “use AI to create AI,” for example by using AI to write GPU kernels. The guests argue that this is not enough. Their test is not whether AI appears somewhere in the workflow, but who the optimizer is and what is being optimized. If humans still supply the central ideas, choose the direction, and judge the improvements, then the process remains humans using better tools.

The hosts place this distinction in a broader technological history. Computers help build better computers; the internet made geographically distributed coordination easier; steam engines could improve coal movement and thereby help build better steam engines. The guests accept those as autocatalytic effects. Their point is that the human brain generating the improvement was not itself the object being optimized. The new possibility, they argue, is that the model or agent brain can become the thing being improved. The optimizer and the optimized system begin to overlap, creating a loop in which cognitive improvement can be reinvested into later improvement.

The conversation then turns to what such a loop could imply. One guest frames capability improvement as a rate driven by a numerator and a denominator: performance on top, and cost, difficulty, capital, energy, or other resources underneath. If the numerator stays fixed while each next step gets harder, progress tends toward a plateau. If the system being improved also becomes better at improving, progress may remain steady. If the numerator grows with a higher power of capability, exponential takeoff is mathematically possible. But the hosts and guests repeatedly resist the easy jump from “closed loop” to “takeoff.” A nonlinear feedback system may converge, diverge, oscillate, or be dominated by noise. The guests say the answer is empirical, especially because deep learning itself lacks a complete learning theory and RSI adds several more layers on top of it.

3. Core Views: Reasoning, Examples, and Limits

The episode’s most useful contribution is its insistence that RSI should be treated as a systems question, not a branding phrase. In the guests’ definition, the crucial distinction is whether the optimizer and optimized object form a real closed loop. Humans using AI to write code, improve kernels, or run experiments may be valuable and may accelerate AI development, but if the central cognitive work still comes from human brains, it remains closer to historical autocatalysis. The stronger claim begins when the model or agent brain itself becomes part of what is being improved, and the improvement is reinvested into the next round of improvement. That is why the guests emphasize the closed loop: if a human must keep filling the key cognitive gap, the system has not yet become RSI in their sense.

The episode is careful, however, not to equate a loop with guaranteed takeoff. Andreessen and Horowitz push on the difference between drawing a feedback diagram and knowing the convergence properties of a nonlinear control system. The guests accept the point. They say the dynamics cannot be known by speculation alone, because even deep learning lacks a fully understood learning theory, and an RSI system layers agents, evaluations, tools, training, and deployment on top of that already empirical foundation. Scaling laws are mentioned as the nearest thing to theory, but they too are empirical laws. The reasoning is therefore conditional: a closed loop creates a possible mechanism for compounding, but whether it plateaus, oscillates, slowly improves, or accelerates sharply depends on the specific system, metric, task, and resource accounting.

The cost denominator is the second major discipline in the conversation. The guests define progress as performance over cost, difficulty, capital, energy, or related constraints, and they explicitly include human hours, experiments, and mental capital in that denominator. This prevents a common category error: mistaking “we spent more money and got a better result” for recursive self-improvement. Maker Maker’s internal culture, as described in the episode, tracks cost, time, and resources, even with an internal leaderboard for experiment cost. That example is partly playful, but it points to a serious evaluation principle. A claim about RSI should not report only model capability; it should report what had to be burned to get there.

The NanoGPT/GPT-2 example clarifies why denominator improvements are important but not sufficient. The guests say that training GPT-2 used to be expensive and slow, while now people can retrain it on a single GPU within hours under the same hardware and data conditions, because algorithmic techniques, data techniques, and kernels improved. They reject the host’s first interpretation that this is merely Moore’s law, because the hardware is held constant. But they also agree that this is denominator improvement, not necessarily numerator improvement. It shows that the same physical resources can do more. It does not by itself show that the system has become better at generating the next improvement.

The guests’ organizational claim is also central. Maker Maker’s RSI approach is not a fully automatic single digital agent replacing everyone. It is a hybrid multi-agent system in which human researchers and engineers are special cases of agents. The unnamed cofounder says good researchers and engineers bring value that is very hard for agents to replace, so the company is treated as humans cohabitating with AI agents. This complicates simplistic narratives about “AI replacing humans.” It also sharpens the measurement burden. If humans remain inside the system, then their judgment, time, and cognitive load must be counted when evaluating whether the system is truly becoming more efficient.

A third view is that RSI must be full-stack rather than local. The guests argue that prompts, harnesses, weights, and even chips can all be recursively improved, but these components are not isolated. A better harness may change what weights need to do; weight updates may still require harness work; local gains can harm the whole. Their preferred method starts with a grounded mathematical framework that maps the space and turns many apparent systems into more general abstractions. Without that, they say research becomes like a collection of specimens: ideas inspired by human behavior that may look locally better but do not prove the overall system is improving. This is why Maker Maker says it began with capital-efficient harness work and then moved into weight updates, while aiming to optimize all levels of the stack.

Inference is their chosen proving ground because it is close to AI itself and economically important now. If a serving stack becomes faster and cheaper, users who want to deploy the latest open-weight models benefit immediately. More importantly for the RSI argument, faster inference lets the system sample, run, evaluate, and attempt improvements faster, so it may improve both numerator and denominator. When the hosts ask how one would speed up KimiK3 inference, the guests say their system does not rely on one LLM. It uses many LLMs for different tasks, allows the model or agent to choose which model and component to optimize, and tries to stay hands-off. They also say the master prompt itself has been improving, and that the component doing improvement and the component deciding what to improve are both dynamic and linked.

The strongest claim in the episode is also the one requiring the most caution. The guests say they are seeing “glimmers” that numerator can rise while denominator is fixed or decreasing. They frame this as empirical measurement, not belief, but they do not disclose detailed results in the episode. They say they have not released findings because they want high confidence. They also report that many external RSI papers or announcements do not work when tested in their internal infrastructure, at scale, on benchmarks, or on real economic problems, and that some apparent breakthroughs turn out to be noise or cherry picking. The claim should therefore be read as the guests’ internal experience, not as independently established fact.

That caution leads to the episode’s detailed discussion of variance, noise, and reward hacking. Maker Maker has an engineer nicknamed the “viceroy of variance,” whose job is to attack results and find what is wrong. The guests say that for new benchmarks or economically valuable problems, they still need “quality time”: hundreds of experiments, close inspection, intuition-building, variance analysis, and checks for strange reward hacking several layers down in agent construction. The hosts add that nonlinear systems may oscillate. The guests agree and add that LLMs are highly stochastic, and that even physical differences between B200 GPUs can show up in high-level behavior. Measurement is therefore not a footnote to RSI; it is one of the core technologies.

Finally, the guests put hard boundaries around acceleration. Intelligence, they say, is not a single scalar but a multidimensional vector. Some dimensions may take off, while others plateau or improve slowly. They use an Amdahl’s-law-style argument: even if the digital part of a workflow becomes a thousand or a million times faster, overall speedup remains limited if that part is only a fraction of the workflow. Coding and mathematics are more favorable because evaluations can be fast and cheap. Drug discovery is less favorable because ideas still require wet-lab experiments and clinical trials that can take months. The host’s closing summary captures their position: they are pursuing actual RSI in which the system improves the full system, including harness and weights, and they claim early signs that performance and cost can improve together. The limitation is equally important: the episode establishes the guests’ framework and claims, not external verification of those claims.

4. Learning and Application

The first practical lesson is to stop treating every “AI helps build AI” story as recursive self-improvement. A better diagnostic is to map the optimizer and the optimized object. Who proposes the improvement? Who runs the experiment? Who judges the result? What is changed in the next iteration? If humans are still supplying the main cognition, the system may be excellent engineering, research automation, or autocatalytic acceleration, but it is not necessarily RSI under the guests’ definition. This distinction matters for investors, founders, and technical leaders because the same sales phrase can describe either a workflow tool or a genuine change in the improvement loop.

The second lesson is to write down the denominator before believing the numerator. Teams evaluating AI progress should not ask only whether benchmark scores rose, release cycles shortened, or inference became faster. They should ask how much capital, GPU time, engineering labor, researcher judgment, experiment count, failed search, and mental load were spent. Maker Maker’s described habit of tracking cost, time, and resources gives a useful management pattern: make the resource ledger visible. Otherwise, a team can accidentally turn expensive trial and error into a story about intelligence improvement.

The third lesson is to separate denominator optimization from numerator optimization. The GPT-2 example is valuable precisely because it shows how much can be gained from algorithmic, data, and kernel improvements on the same hardware. That kind of work can produce enormous practical value: lower training costs, faster serving, more rollouts, and cheaper evaluation. But it mainly proves that the same physical resources are being used better. To claim RSI, a team needs additional evidence that the system is becoming better at producing improvements, not just cheaper at executing known improvement methods.

The fourth lesson is to design against local optimization. Prompts, harnesses, weights, data pipelines, kernels, and hardware interact. A better prompt can expose harness weaknesses; a harness change can alter what a model update means; a kernel gain can change the economics of search. The guests’ emphasis on a unified framework is useful because it forces a team to ask whether a local gain improves the whole system, whether it transfers to real economic problems, and whether it creates a new bottleneck elsewhere. In practice, that means evaluating system-level outcomes, not just isolated demos.

The fifth lesson is to keep humans in the architecture honestly. The episode’s multi-agent framing treats human researchers and engineers as agents inside the system, not as embarrassing leftovers to hide. That is a productive way to design AI organizations. Agents can search, generate, test, and execute, while humans may still contribute problem selection, taste, anomaly recognition, and framework-building. The boundary condition is that human involvement must be counted. If expert judgment is what makes the system work, it belongs in the denominator.

The sixth lesson is that measurement is not a secondary concern. Closed-loop systems can plateau, oscillate, amplify noise, or reward-hack their own evaluations. A credible RSI program needs repeated runs, control comparisons, variance analysis, adversarial result review, cost tracking, and failure analysis. The “viceroy of variance” anecdote is funny, but the principle is severe: the more dramatic the self-improvement claim, the more someone should be trying to break it. Best-case demos are weak evidence unless they survive repeated measurement and attempts to falsify them.

The seventh lesson is to choose application domains by feedback speed and bottleneck structure. Inference optimization, coding, and mathematics are more plausible early RSI arenas because evaluation can be fast, cheap, and largely digital. Drug discovery, robotics, manufacturing, and other atom-moving workflows face slower loops. Even if AI makes the digital portion vastly faster, the whole workflow may still be gated by wet labs, clinical trials, physical experiments, safety checks, or deployment constraints. Before applying RSI language to a domain, ask where the slowest component is and whether the feedback loop can actually be shortened.

The safest conclusion from the episode is not that RSI has already produced open-ended takeoff. It is that the test for RSI has become more concrete. A system would need to show sustained stronger improvement under the same or lower resources, with results that survive noise, variance, reward hacking, metric ceilings, and hidden human costs. If better outcomes come from more money, more people, or more uncounted experiments, the work may still be valuable, but it should not be marketed as a closed-loop self-improving system.

Source

• Original episode: Recursive Self-Improvement and the Future of AI