Google and NVIDIA spent Google Cloud Next last month pitching the same idea from different angles: serving AI is getting cheaper, and they’re the ones doing the cutting. Google announced new chips designed specifically for serving AI to users, separate from the chips used to train models, a sign that running AI at the user-facing layer is now a distinct enough cost problem to deserve its own hardware. NVIDIA, partnering with Google on a new generation of cloud machines, claimed up to 10x lower cost per AI response and 10x more responses per unit of electricity compared to the previous generation.

Last week, I wrote about the infrastructure providers behind Frontier AI and the over $100 billion deals that Anthropic signed with AWS, Google, and Broadcom, which are shaping the future of frontier AI technology. That piece was the macro view: who controls the compute, the chips, and the power contracts that frontier AI runs on. What cheaper inference does, and doesn’t do, for the people doing the work is the micro level of this issue.

So why do operators, managers, and students still feel buried? Because cheaper inference doesn’t automatically translate into less friction in your day.

📡 Going live this week. RAG is the engine behind every enterprise AI tool you’ve already used and trusted. Glean. Copilot. Notion AI. The internal assistants your company is piloting right now. It’s also the thing nobody outside the data team is talking about. That’s a problem, because if you’re a manager or operator and you don’t understand RAG, you can’t tell the difference between an AI rollout that earns adoption and one that will be sundowned in six months. I’m going live on Substack this Wednesday, May 20th, at 8:30 AM EST to break it down: what RAG is, why it’s the foundation of every useful enterprise AI deployment, and why operators (not just engineers) need to understand it.
Join Me Here

Back to inference economics.

Cheap Tokens, Same Overwhelm

Gartner expects agentic AI workloads to burn 5x to 30x more tokens per task than standard chatbots, which means the falling per-token price is already being offset by rising consumption. The companies serving you AI will keep a healthy share of those savings, and the ones they pass along will get poured into longer context windows, more tool calls, and more autonomous loops. None of that, on its own, fixes the underlying human problem: the work itself keeps outrunning the worker’s ability to stay in context. Cheaper inference makes it economically viable to throw more AI at a worker without making the work itself any easier to do well. If you’ve felt that the tools got smarter but your workload didn’t get lighter, you’re reading the curve correctly. Cheaper inference is a supply-side phenomenon. It doesn’t reach the worker until something on top of the model reduces friction.

Who Pays for Cheaper Inference

The cost of running AI doesn’t disappear when per-token prices fall. It gets redistributed:

Frontier Labs absorbs some of it itself to keep its models competitive.
Hyperscalers recover it by bundling inference into platform contracts, the same playbook AWS ran with storage and bandwidth a decade ago.
Enterprises pass it through to end users as seat prices, usage caps, and tiered features.

The worker sits at the bottom of that chain. A 10x cost reduction at the chip level rarely translates into a 10x improvement in a worker’s day. By the time it filters through cloud contracts, vendor pricing, and product packaging, your team will end up with a marginally better tool and a slightly larger software budget. The savings are reinvested in additional capabilities for vendors to sell, rather than in capacity that the worker keeps.

What This Means for the Future of Work

The shape of work over the next few years will be decided by who can afford to deploy frontier inference broadly, and by how that inference is packaged before it reaches a worker’s desk. AI capability is becoming an organizational asset rather than an individual one. The worker at a company with a rich inference budget will get more out of frontier AI than one without, and that difference will widen as agentic workloads burn 5x to 30x more tokens per task than today’s chatbots.

Once every team has a frontier model, the orchestration layer will be what separates teams. Whether your context, decisions, and in-flight work are held together by a system or scattered across tabs determines how much of that frontier capability you can actually use.

And the human cost of bad orchestration grows with model capability. More powerful tools used badly create more interruptions, more half-finished threads, more cognitive debt. Cheap inference, poorly wrapped, is a faster way to feel overwhelmed.

Subscribe now

The infrastructure layer is solving its own problem. What still needs building is the layer between cheap compute and a working day, the one that decides whether all that capability turns into leverage for a person, or just more input to sort through. Asaura AI is one bet on that layer, built for people who already feel the difference between having access to a powerful model and having a successful day at work. The broader point applies regardless of which tool you use. In a world where the model is cheap and the work keeps expanding, the system that organizes your context, your priorities, and your decisions is the part that compounds.

Per-token prices will keep falling. Token consumption will keep rising. Both can be true, and both already are. What decides whether that ends up as leverage for you, or as a faster firehose pointed at your inbox, is the orchestration layer sitting between the chip and the chair. I’ll keep following this trend, both on the economics side and on what the layer between the model and the worker has to look like.

Cheap Tokens, Same Overwhelm

Who Pays for Cheaper Inference

What This Means for the Future of Work

Subscribe to The Data Letter for more on the economics of AI and the future of work.

NVIDIA and AI Inference Economics in 2026