Naveen Rao on the Scarce Resources Behind Abundant Intelligence

1. Guest Background

Naveen Rao approaches this episode of 20VC as a serial entrepreneur working on the machinery of AI. Across approximately 73 minutes, host Harry Stebbings asks him to connect technical choices with commercial consequences: whether electricity can support further expansion, where computing efficiency can improve, and which businesses might retain profits as intelligence becomes more widely available.

The supplied guest background identifies Rao as a founder of Mosaic ML whose current venture is Unconventional AI. In the introduction, Stebbings says Mosaic ML was sold to Databricks for $1.3 billion and describes Unconventional AI as having raised hundreds of millions of dollars at a recent valuation approaching $5 billion. Those figures are the host’s account. The material does not specify Rao’s current job title or name a product from the new company.

His relevant experience also emerges through his recollection of starting Nirvana in 2014, when he believed neural networks required a new computing architecture. Today, he describes his team as energy first and willing to change the model itself. That continuity helps explain the interview’s perspective: Rao sees intelligence as something delivered by a physical system, and he is actively building around that belief. His enthusiasm for hardware should therefore be read alongside his direct participation in the field.

2. What the Episode Covers

The conversation begins with a distinction between accessing spare electricity capacity and constructing new supply. Rao cites an industry estimate he has encountered of roughly $50 billion for an additional gigawatt. He also estimates average US grid output at about 900 gigawatts and peak capability at 1.2–1.3 terawatts. His account is that data centers have recently obtained power largely by increasing utilization of existing infrastructure. These are interview estimates, and the difference between average output and peak capability should not be treated as continuously available supply.

Rao expects that headroom to become scarce within a few years if AI demand continues growing. Expansion would then converge toward the pace of construction. He doubts that adding 40 gigawatts annually is currently feasible; even 10 would be impressive and, using his cost assumption, imply approximately $500 billion. His argument concerns a physical constraint on growth, rather than a shortage that financing alone immediately resolves.

He nevertheless interprets contemporary financing pressure as a temporary disturbance rather than a bursting bubble. His premise is that solving AI’s remaining technical problems will reveal additional uses. He also reports anecdotal evidence of data centers paying electricity premiums. His illustration of compute revenue at five times electricity expenditure, allowing a 30–40% power premium, explains possible willingness to pay; it is not a disclosed operating benchmark.

Computing efficiency provides his proposed response. Rao separates the physical, architectural and model layers. In his assessment, physical scaling has slowed substantially, while recent gains have depended heavily on lower precision, moving from 32-bit toward 16-, 8- and 4-bit computation. He sees further opportunity in locality and reduced data movement. Sparsity illustrates why the layers must be designed together: removing connections can help on a GPU, but off-chip memory may limit the benefit. Purpose-built structures might exploit the same idea more fully while still supporting general intelligence.

Training introduces another resource question. Rao emphasizes curated data, loss functions and the sequence of training stages. He expects the provenance and method of specialist teaching to become valuable, potentially giving specialized data and curricula defensibility. Capability, in this account, depends partly on how expertise is constructed.

Robotics extends the thesis into physical work. Against Stebbings’s concern about timing, Rao forecasts usable prototypes for some tasks in two to three years and a market in five to seven. He also predicts a $10 trillion inference market within a decade and robotics at potentially 100 times agentic coding. No quantitative model accompanies those estimates. He favors the robotics technology stack, expects possible integration of models and devices, and focuses on translating tasks into motor output. His preference for technical progress over early market validation also meets a practical objection from Stebbings: investors still face constraints around valuation and ownership.

The closing scientific vision is broader than a product forecast. Rao hopes AI will expand the set of questions humans can investigate. Wormholes serve as an example of theoretical questions to explore, not as a claim that AI has established the phenomenon or made it engineerable.

3. Core Views: Reasoning, Examples, and Limits

A useful distinction in Rao’s reasoning is that improving capability and improving resource efficiency are separate achievements. He considers reinforcement learning less efficient than pretraining when measured as improvement per unit of compute or money, yet expects it to remain important for several years. An expensive method can still be worth using when it improves a task that matters. His enthusiasm for RL therefore should not be translated into a claim that it is the cheaper route to every capability.

His tennis example makes the mechanism concrete. A learner develops an idea of a good serve and receives feedback from ball speed and whether the ball lands in bounds. Such outcomes provide something against which actions can be evaluated. Rao argues that collecting demonstrations cannot cover every task in the world. The limitation is equally consequential: tennis offers relatively clear measurements. The interview does not establish that every professional objective has an equally reliable reward signal, or that optimizing an available measure necessarily captures the desired outcome.

Rao also questions the efficiency of Transformer reasoning organized around repeated sampling, comparing traces and feeding traces back into a model. He expects alternatives, but supplies neither a replacement architecture nor comparative results. His account of innovation incentives is similarly interpretive. He argues that frequent release competition limits major laboratories’ appetite for paradigm risk, while xAI’s rapid pursuit of the frontier demonstrates the force of resources, talent and execution. Startups might consequently do more exploratory work. That explanation does not establish that large laboratories lack deeper innovation, or identify where the next successful architecture will originate.

The most tangible productivity example concerns coupled oscillators. Rao wanted an initial understanding of how noise would degrade information across four to six hops in a daisy-chain arrangement rather than all-to-all connectivity. Previously, scheduling the work, writing code and checking it might have taken a research team around two weeks. With a model generating code, Rao inspecting it and several rounds of interaction, he says he obtained an answer in about an hour. The achievement is faster preliminary exploration under expert supervision. It is not evidence that an entire research program can be compressed by the same ratio.

That qualification leads to his strongest organizational argument. Generating more artifacts can shift the constraint toward verification. Rao describes a complete process: establish a useful specification, verify intermediate outputs, then confirm that the final implementation satisfies the specification. A checking layer alone does not establish control over the whole process. Correct code can still address an unhelpful question. He also warns that selecting among model-generated solutions remains insufficiently understood and that a long chain of decisions can look robust while being wrong. Anticipated future trust in agents does not resolve those present limitations.

His commercial thesis makes a parallel distinction between demand and profit. Some intelligence, he argues, is already becoming a commodity because older open models remain useful. Inference operators add deployment quality, availability and cost optimization, but falling prices per token could squeeze the margin available in that layer. He prefers hardware because improvements in converting energy into intelligence may still be substantial. His definition includes chips, systems, cloud and networks. The interview does not quantify how much of an efficiency advantage any supplier could retain as profit.

His open-versus-frontier forecast illustrates why volume is an incomplete measure. In three years, he expects open models to carry around 80% of tokens and frontier models 20%, with spending roughly divided equally. Routine tasks could be cheap while a small number of important problems justify premium prices. This is a forecast about a possible structure, not an observed market allocation. His skepticism toward coding-agent wrappers has a narrower evidentiary base: employees at his company switch tools fairly freely. That observation motivates testing the independent value of an orchestration layer or IDE; it does not establish switching behavior throughout the market.

Rao’s criticism of conventional SaaS becomes more specific when he discusses outcome delivery. He supports combinations of software and services that use AI to generate work and humans to verify it. Harvey is his legal example. His willingness to send it roughly 80% of his own legal workload depends on gaining confidence in its NDA and contract-review capabilities. That is conditional purchasing interest, not a demonstrated replacement rate. He also expects large institutions to have more capacity for customization, while smaller organizations often need ready-to-use services.

Finally, he argues that cheaper production changes demand. A hypothetical company could keep its $10 million software budget while demanding much faster fixes and releases. His construction example similarly imagines robots performing 80% of tasks, shorter schedules and more projects. These examples explain why output might rise without an equivalent fall in combined human-and-AI spending. They do not establish future employment, construction schedules or budget shares. Their analytical value lies in asking how customers respond to lower costs, rather than treating the illustrative numbers as results.

4. Learning and Application

For infrastructure planning, use Rao’s argument to separate utilization gains from newly constructed capacity. Record sustainable supply, construction progress and capital requirements separately. Then examine a scenario in which demand keeps growing while spare capacity declines. His quoted cost per gigawatt can motivate sensitivity analysis, but it should not become a universal project budget. Likewise, peak capability should not be counted automatically as continuous supply. The practical question is whether a project can operate under the supply conditions it will actually face.

For engineering teams, turn co-design into a comparison under consistent task-quality requirements. Examine precision, memory access, data movement and connection structure together. If sparsity reduces arithmetic without reducing movement, investigate where the expected system benefit disappears. Specialized hardware may offer greater gains, but the interview provides no transferable savings percentage. Training choices require the same discipline: compare resource expenditure, target-task improvement and reward quality independently. RL’s usefulness does not establish superior cost efficiency, and an easily measured reward still needs to represent the intended result.

For knowledge work, the most directly usable idea is the verification loop. Define the question and acceptance conditions before generation, inspect consequential intermediate work, and test the final artifact against the original goal. Begin with bounded exploratory tasks whose reviewer can understand the output, as Rao could inspect his noise-modeling code. Measure time to an initial answer separately from time to a dependable deliverable. If review cannot keep pace, reduce simultaneous production or narrow the task. Otherwise, faster generation may simply create a larger queue of unchecked work. An impressive preliminary answer should not silently become a promise about complete research delivery.

For business analysis, keep token volume, customer spending, operating costs and profit allocation separate. A coding tool should demonstrate what users retain it for when the underlying model changes. Observe switching behavior and distinguish model capability from the contribution of the interface and orchestration. An outcome provider needs explicit acceptance criteria, an identifiable reviewer and an account of the work that must return to the customer. Large buyers may justify customization; smaller buyers may prioritize immediate usability. Rao’s Harvey example helps frame these questions but supplies no verified customer savings.

The investment discussion adds a method for examining technical judgment. Rao recalls Steve Jurvetson discussing numeric representations, annotating papers and asking follow-up questions. A practical adaptation is to ask a team to explain one consequential design choice, show its evidence and identify what finding would make it change direction. This tests how the team reasons, rather than merely how persuasively it presents a destination.

Stebbings’s space-investment dilemma supplies the boundary: a $20 million check from a $400 million fund creates 5% exposure even when the team seems excellent and the investor does not understand the technology. Rao recommends investigating the team’s ability to correct assumptions, pivot and make technical decisions, supplemented by conversations with people who know them. Two or three meetings still leave uncertainty. Technical enthusiasm cannot make that uncertainty disappear.

Finally, translate robotics and scientific optimism into staged questions. Can a prototype perform the specified task? Can a task description produce dependable movement? Does expanded research activity yield a verifiable finding? Technical progress can inform a judgment without validating a valuation or investment return. In science, distinguish posing a theoretical question, establishing whether a phenomenon is real and engineering an application. Rao’s wormhole example expresses the ambition to widen inquiry; it does not provide evidence that those later stages have been completed.

Source

• Original episode: Inference Will Be $10TRN Market: What No One Sees | Software Investors are F***ed | Naveen Rao