AI’s next hard problem isn’t parameter count. It’s intelligence per watt.

At two in the morning, a laptop closes its lid while a quiet agent keeps sorting meeting notes, filing photos, triaging email, and turning a workday into tomorrow’s list. Once that becomes ordinary, the question around AI stops being only “how smart is it?” It becomes: how much power, bandwidth, money, and privacy does that intelligence consume?

For two years, the industry has trained its attention on bigger datasets, longer context windows, and stronger benchmarks. Capability matters. But it is only the first half of the story. The harder half is making good-enough intelligence run for long stretches, cheaply, on enough devices—rather than sending every request through a distant data center.

Stanford researchers Avanika Narayan and Jon Saad-Falcon use a useful phrase for that problem: intelligence per watt. How much useful intelligence does a system deliver for every watt it draws?

That is not a single chip-spec number. The numerator is task performance: whether a system can actually write code, process documents, answer questions, or complete an agent workflow. The denominator is the real power cost of the whole stack: model, quantization, batching, inference engine, kernels, memory, hardware, and deployment. AI efficiency is not won by one model or one processor. It is the accumulated result of dozens of engineering decisions.

Not every task deserves the frontier model

Cloud AI today rests on a quiet assumption: when a user asks a question, the strongest centralized model should handle it. It’s convenient. It’s also expensive.

A large share of daily work does not need frontier-grade reasoning. Filing documents, searching personal material, doing first-pass classification, rewriting structured text, and handling repeatable business processes often care more about reliability, speed, data boundaries, and near-zero marginal cost. Sending all of that to the cloud is not always buying a better answer. Sometimes it is simply taking the most expensive route through an ordinary task.

The case for local models is not that the cloud disappears. It is that workloads get separated. Keep what can stay on the device on the device. Escalate to the cloud when a task genuinely needs more capability, outside knowledge, or heavier computation. It is a simple systems principle: local by default, hybrid by design.

That changes more than a budget line. Data does not have to leave the device on every call. A system can still do useful work when connectivity is poor. Individuals and companies do not hand every trace of their work to one vendor. The point is not an “offline” badge. The point is putting choice back with the user.

Companies shouldn’t only rent tokens

The asset businesses most often undervalue in AI adoption is not a GPU. It is their own knowledge.

Customer conversations, operating records, internal documents, edge cases, and the reasons behind decisions add up to a company’s real capability. When every log, query, and workflow is sent straight into an external black box, a company may gain short-term speed while steadily giving away its context.

That does not make cloud services off-limits. The problem is treating “plug everything in now, migrate later” as the default path. Migrations do not happen by themselves. Data formats, permissions, toolchains, staff habits, and cost structures quickly become tangled. The easy choice at the beginning becomes lock-in later.

A more practical approach is to divide work into two groups.

The first is explorer work: novel, unstable, high-uncertainty tasks where maximum capability is valuable. Frontier models make sense there. The second is exploit work: mature, repeated processes with clear evaluation criteria. Those are the places to test open-weight models, smaller models, or owned inference early. The first step does not have to be buying servers. Redirect part of the workload, compare quality, latency, cost, and data boundaries, then decide what deserves to be brought in-house.

It is not the flashiest route. But it shifts a company from merely buying intelligence to building the ability to use it. The model is one component. Without alignment across applications, tools, memory, access controls, compute, and employee workflows, even a great model is just an expensive call.

Efficiency will not arrive by itself

“Compute always gets cheaper” is often treated as a natural law. History says otherwise.

Computing became ubiquitous through concrete breakthroughs in transistors, compilers, architecture, manufacturing, and software engineering—not because a curve politely bent toward lower cost. AI will be the same. Model architecture, training methods, data quality, inference kernels, memory scheduling, device accelerators, and deployment strategy can all produce the next order-of-magnitude gain.

That is why AI competition will not end with whoever trains the largest model. It is becoming a layered market. Some systems will optimize for maximum capability; others for minimum latency, stronger privacy, lower unit cost, or a productive local-cloud split. There will not be one winner across models, chips, and applications.

The real mistake is assuming all future demand will be served in one way. If many workloads can be handled by smaller, cheaper systems closer to users and their devices, pricing and moats built around centralized token rental will have to be recalculated.

AI will not reach everyone merely because parameter counts keep rising. It has to pass through batteries, thermal limits, bandwidth, bills, and privacy—the less glamorous parts of reality. The teams that can fit good-enough intelligence into less energy and nearer devices will be the ones that move AI from the demo stage into everyday life.