AI Semiconductor Endgame (IV)

What does the rise of open-source models mean for semiconductors? Will it impact semiconductor revenue?

This has probably been one of the most hotly debated questions in the market over the past few months.

Open-source models may already account for more than 20% of token volume, and that share is growing faster. Their prices are a fraction of what closed-source flagships charge. Every new open-source release sends the market back to the same calculation: if tokens keep getting cheaper, can model companies still make money? And if they can’t, who’s going to buy all those GPUs?

Seen statically, it really does look like a contradiction:

On one hand, cheaper models bring more people to AI, so we need more compute. On the other, open source pushes down OpenAI’s and Anthropic’s prices, and if the model companies can’t make money, who pays for the enormous CapEx of the next few years?

I. Will open source hurt semiconductors?

Unpack the market’s worry and it’s really two different claims.

Claim one:

open source will drag down even premium token prices. That would hurt the ARR of the two closed-source leaders, OpenAI and Anthropic, and weaken their ability to fund compute purchases by raising capital and borrowing against future earnings.

Claim two:

open source wins the market with price wars and takes too much share, but its own revenue and margins are too low. The model layer as a whole ends up with less to spend on semiconductors, and the open-source players’ own fundraising and growth slow down too.

An open-source token is still a token. It runs on real compute, and at a similar model size it actually uses a bit less, thanks to optimizations like DeepSeek’s sparse attention. But for semiconductors, open vs. closed isn’t what matters. What matters is whether the people running the GPUs are making money. Gross margin is the most important factor behind the rush for chips, almost without rival.

When margins are high enough, every GPU you can grab makes you money. The more you buy, the more you make.

At 70%+ gross margin, a GPU pays for itself in a year, and every GPU is a money printer. Even at just 50%, you can still afford to take on more financing risk and borrow to buy more.

So the question becomes: will open source drive gross margins down?

Take a static view of token prices, and the answer seems to be yes: prices keep falling. But a static view is the wrong lens. Token economics needs a dynamic framing.

II. The dynamic equilibrium of token economics: costs fall as fast as prices

Token prices keep falling. That’s a fact. The entire industry is pushing costs to fall just as fast. That’s also a fact.

Margins hold. Costs fall. Prices fall. Usage rises.

This is the central thesis of this piece: with every new generation of models and every new generation of hardware, the system settles into a new equilibrium. Call it the dynamic equilibrium of token economics.

So don’t take a static view. Exponential token deflation is the ever-moving backdrop of the AI era.

As I’ve said before: in the AI era, token efficiency improves by six orders of magnitude per decade, three from software/algorithms and three from hardware.

Jensen said it back in 2023: every decade, hardware and software get a million times faster, and that millionfold comes from progress on every front: interconnect, memory, packaging, PCBs, architecture and manufacturing.

That’s 1,000x per decade from software and 1,000x from hardware.

The past decade bore this out. Break hardware’s 1,000x down: process technology gave about 3x, complex instructions like DP4/HMMA/IMMA about 10x, lower-precision number formats 8–16x (FP32 -> BF4, MX4), and sparsity optimizations helped on top of that.

Looking at hardware today, that pace is still holding. HBM bandwidth and capacity upgrades are accelerating. Advanced packaging keeps introducing new 3D architectures. Optical interconnect bandwidth is rising exponentially while latency and power consumption fall. Power standards and cooling technology are advancing rapidly too. All of this keeps the long-running backdrop in place.

In concrete terms:

1. Every hardware generation doubles token throughput, cutting the price per token in half. Blackwell delivered as much as 4x the throughput of H100. Vera Rubin has just started testing, and from what was shown at GTC and Hot Chips, it should roughly double token throughput again.

2. On top of that, for the same model generation, software cuts the price in half again, sometimes by three-quarters, every year or even every six months, as KV cache compression and all kinds of model-level optimizations keep improving.

Tokens themselves are getting more valuable, too. The 5.5 generation made a big leap in token efficiency: the same task takes fewer tokens, while the quality of each token keeps getting better.

As long as there’s competition, everyone will race to deliver the same quality at lower cost, so the gross margin on incremental token sales is sustained mainly by continuous hardware and software optimization.

Even open-source models can keep margins at a very healthy level.

Per-GW unit economics: the hardware ledger

It’s easier to see if you run the numbers on a single GW. The market may be underestimating how high token margins are today.

On the latest B300s, frontier flagship models run inference so efficiently that 1GW can generate more than $80B of token revenue a year. With Vera Rubin, token throughput per watt rises at least another 50–100%, taking revenue per GW to $120B or more. Gross margin per GW will keep rising.

Of course, actual revenue next year may not reach those levels. The two labs still compete with each other, so flagship token revenue may stay around $80B per GW. Even so, flagship inference margins stay at 75%, even above 80%. Anthropic’s inference revenue this year is expected to hit $50B/GW, and next year even its blended figure could rise to $80B/GW. At margins like these, nobody hesitates to grab compute.

Open-source token margins aren’t as high: at official list prices, roughly 60–70%. Annualized revenue per GW can still exceed $50B.

Lately, Sol 6.1 and Opus 5.5 have set such an aggressive kill line (a price-performance bar that anything weaker must undercut or die) that open-source models have had to cut prices hard, putting their margins under enormous pressure. But I think the squeeze is temporary. DeepSeek, for example, recently optimized Flash 4.1, and its margin climbed back above 70%.

Now the cost side. Amortize today’s hardware over 6 years and it’s only about $8B/GW a year. Revenue is ten times the hardware amortization. That’s why short-term neocloud contracts have recently fetched a staggering $40–50B/GW a year: the tokens that capacity produces really do sell for even more.

Even if hardware prices keep rising, this equilibrium won’t break easily. The huge gap between token revenue and hardware cost is the pie the AI infra/semiconductor civil war will be fought over. That war is the subject of my next piece.

This dynamic equilibrium has been in place since 2023.

GPT-4’s price fell 10x a year, 100x in just over a year. Each new flagship kept its pricing power on the strength of its performance, at around $50 per million tokens, while the previous generation was marked down hard.

Token deflation is nothing new. Yet the game has only gotten healthier: each token delivers more productivity and value, ROI keeps improving, and that has boosted both adoption and usage.

Smartphone chips followed the same path in the mobile internet era.

So treat falling token prices as the norm, a backdrop that’s always moving. Even when prices drop, even by half, cost cuts can largely keep the equilibrium intact. And lower prices bring more usage, which grows total revenue.

For AI infra, the biggest payoff of this equilibrium is that it protects gross margins, and gross margin is the single biggest thing keeping the industry hungry for hardware.

III. Value-tier tokens are a commodity, premium-tier tokens aren’t: two entirely different markets

The dynamic equilibrium answers whether price cuts will blow through margins. But will open source hurt the most profitable part of closed-source revenue, the flagship tokens?

I don’t think so, because the value tier and the premium tier are two entirely different markets.

The value tier is a commodity market, and everyone is in a brutal race to the bottom. As soon as one player draws a new kill line, everyone else has to match it. Official list prices may look unchanged, but the prices hosts charge on OpenRouter adjust in real time. Anything in the value tier is easily commoditized: products, even human labor.

The premium tier is a different story. Competition here is confined to OpenAI and Anthropic, which limits commoditization. Open source can’t touch premium-tier pricing, which stays relatively stable.

We’ve seen this setup once before, in the mobile internet era:

Qualcomm built some of the best phone chips in the world. But you can’t stop technology from diffusing: MediaTek and the whole East Asian supply chain, China included, quickly produced similar chips at a quarter of Qualcomm’s price, or less. (This is a lot like distillation today, which probably explains 30–50% of why the leaders can’t open up a bigger lead over open-source models.)

Qualcomm’s defense was to keep selling its premium chips at high prices for flagship Android phones, while creating lower-tier versions of those same chips by disabling a few cores, lowering clock speeds and trimming specs. That let it build midrange and value-tier chips at very low cost and sell them for less than MediaTek’s. Across many generations, Qualcomm’s value-tier chips outperformed MediaTek’s flagships while costing less.

Bring that story into today’s AI market, and the parallels are almost exact. Open-source players use much cheaper distillation to build value-tier models that are noticeably less capable than the flagships, then try to win by undercutting them on price. What OpenAI and Anthropic are doing is exactly what Qualcomm did: distilling their own premium models into offerings that are cheaper and more capable than their open-source rivals, setting the kill line and keeping the fight confined to the value tier so it never spills into the premium tier.

So the premium-tier war is fought only between the two of them, and Astra can still charge a lofty $50 per million tokens. The closed-source value tier serves a second purpose too: it keeps open source from taking more of the market and from piling up too much profit. The less profit open-source providers make, the harder it becomes to fund the compute they need to catch up.

The latest round of kill lines, GPT-6.1 Sol and Sonnet 5.5, has put enormous pressure on open source. But this is just one turn in a turn-based game: open source keeps improving, and its next generation is still being trained. Exponential token deflation has been running in the background the whole time; open source has only made this year’s value-tier price curve a bit steeper. And although distillation has recently gotten harder, I still think open source can stay roughly six months of the frontier.

There’s been a price war in premium tokens lately too. Both labs are heading toward IPOs, and they may give up some premium margin for now to make ARR look better, land the next round, or get to a smooth IPO. But a destructive price war is unlikely from here. This is a duopoly.

Why won’t premium become a commodity?

On the supply side: the LLM era has no network effects, but it does have a strong RSI (recursive self-improvement) effect. Each generation’s model produces the data that trains a stronger next generation, and that loop keeps getting stronger. The leader’s advantage reinforces itself.

The demand-side reason comes later: demand for flagship models won’t let up.

So “open source threatens closed-source revenue” is only half right. It threatens the closed-source value tier, where the incumbents’ defenses are highly effective. The two markets are separate, and you can’t lump them together.

IV. How much does open source actually matter to semiconductors overall?

That covers the dynamic equilibrium (open-source margins can hold up too). Which leaves the most basic question of all: how big is open source, really?

Open source may already be 20% or more of tokens. But measured by ARR, actual customer spending, its share is only 5–10%. That revenue, together with the gross margin it earns, determines how many GW of compute it can support. That is what tells us how much open source matters to semiconductor demand.

In other words, in compute terms, open-source token revenue supports only about 5–10% of the associated AI infra/semi revenue. Include in-house use by companies such as Meta and Google, and the share is smaller still. As I covered last time, North American open-source model revenue is still under $10B, while closed source combined is above $120B, probably near $140B by now. For the foreseeable future, open-source ARR will grow faster than closed source, but not dramatically faster.

Now back to the two worries from the start.

The first, that open source drags premium token prices down, doesn’t fully hold. The two markets are walled off from each other. Open source can’t force premium prices down, and the bulk of ARR sits in the premium/high tier. That said, high-tier models like Opus and Sol may take some hit right after each new open-source release.

The second, that open source’s own margins are too low and drag down the model layer’s ability to buy GPUs, doesn’t hold for now. Open source running on B300 still earns 60–70%+ margins, and its share is small. Its own revenue constraint hasn’t kicked in either: without decent revenue you can’t keep training, and the iteration loop is hard to sustain. But open source is growing strongly right now, and the loop is turning just fine.

So the conclusion: open source won’t drag down gross margins. And only secure margins keep the appetite to buy secure. Even if margins take a hit, as long as open source keeps 30%+ and closed source keeps 60%+, the game works just fine. Stronger open source won’t mean weaker demand for semiconductors and infrastructure.

As long as the impact on margins stays limited, open source gives Nvidia a more diverse customer base. Nvidia wants to commoditize the model providers immediately downstream in the supply chain, giving itself and infra more bargaining power. That actually makes open source a positive for AI infra/semi.

As for the idea that open source will eventually take the vast majority of revenue, I think it’s unlikely, for reasons I gave last time: mainly, open-source compute can only grow so fast, and the closed labs’ value-tier tokens are fully competitive.

V. The equilibrium’s precondition: usage has to outrun price cuts

So far, the dynamic equilibrium looks solid. But it rests on one important precondition.

Today’s high token margins depend heavily on a compute shortage. So will compute stay scarce forever?

Whether margins hold ultimately depends on how big token demand can get, in other words, whether total ARR can keep growing at its current pace.

The Jevons paradox has played out on OpenRouter countless times: prices fall, demand grows by even more, and total revenue goes up. That said, recent data suggests prices are falling so fast that demand can’t keep up in the short run, so total revenue growth may slow.

One study using OpenRouter and Azure data put the short-run price elasticity of token demand at roughly 1: cut the price 10% and usage rises about 10%, leaving total spend basically flat.

So after all that, the question of open source’s impact on semiconductors comes down to this**: can token demand keep outrunning price cuts?**

VI. Token demand will keep growing massively, and the value tier won’t easily replace demand for flagships

There are a few reasons:

1. Token usage is heavily concentrated among top users, and they can absorb every increment of model capability

2. “Once models are more capable than most people need, further improvements stop mattering” is an illusion

3. Open-ended tasks exist in every field, and on those tasks flagship models are clearly better

4. At its core, capability is about moving up the abstraction stack, so the work a model can orchestrate keeps getting more complex

5. As AI makes information easier to access, insight easier to come by, and intent easier to turn into action, these gains combine to open markets an order of magnitude larger at each stage

1. Better models primarily benefit power users, and continued gains in capability keep unlocking more usage

The top 10% of users account for most of the usage. The top 1% use about 10x as much as the top-10% tier on average and 100x the median user, and they’re usually the ones who get the most out of what models can do.

What better models mainly improve is the ability of the top 10% of power users to run long-horizon tasks. Model capability is what caps their usage. As that ceiling rises, their usage keeps growing.

Users in the middle don’t need as much raw model capability. Their main constraint is tool integration: the pieces of their everyday workflows still aren’t well connected.

So token growth may come from different places for these two groups: more capability and long-horizon tasks for power users; better integration (connectors, workflows and personal agents) for users in the middle.

For users in the middle, how that capability is packaged into a product matters more. They may be the users for whom the application layer is most useful.

2. “Once models are more capable than most people need, further improvements do nothing for revenue” is probably an illusion

First, even ordinary users want the smarter model when they already have more intelligence than they need, just as people kept buying phones with more compute than they’d ever use.

Second, in tasks where labor makes up most of the cost, using the flagship may actually save money. When model spend is small next to wages, say labor is 90% of total cost, a stronger model means less human review and rework. It saves more human time and brings total opex down.

There’s another axis: how open-ended the task itself is.

3. Open-ended tasks exist in every field, and flagship models are far better at them

Bounded tasks have a ceiling. Tax filing, for example, has capped complexity; once a model clears that bar, extra intelligence adds no more value. Unbounded tasks have no ceiling. In AI research, there’s always more to do.

First, even in a task like tax filing, where complexity is capped, reliability still matters. Greater intelligence brings greater reliability. A bounded task has a ceiling on complexity, not on reliability.

The costlier a mistake, the more every bit of extra reliability is worth.

Reliability compounds across steps: push each step to 99%, and reliability over 20 steps rises to about 82%. A small gain in per-step reliability turns into a big gain at the task level, and that also means less human review and rework.

When human review and rework make up most of the cost, the stronger model may well be the cheapest option overall.

Second, “unbounded” describes a task, not a field. Any sufficiently open-ended task is unbounded.

Adversarial tasks are unbounded by nature, and open-ended tasks exist in every field, not just AI R&D, software engineering and trading.

The total volume of bounded tasks doesn’t grow much. Unbounded tasks are different: open one up and it keeps spawning new ones. That’s pure incremental demand.

Stronger models do far better here and earn a higher ROI, which is also why revenue took off so fast after Astra launched.

Third, capability also determines how long a single task can keep going. A little over six months ago, a task would stall after two or three iterations; now it can go through a dozen or more and keep going. As capability grows, tasks run for more iterations, and the abstraction layer keeps moving up.

Fourth, agents will increasingly go up against other agents, and flagship models will always hold the edge. In adversarial settings, value depends on relative capability, not absolute capability, so there’s no such thing as a model that’s “good enough.”

4. At its core, capability is about moving up the abstraction stack, so the work a model can orchestrate keeps getting more complex

In the end, a more capable model is a model that abstracts better. The more complex the work a model can orchestrate, the more we ask of flagship models, and the more tokens they consume.

A model’s ability to manage complexity is finite. It needs layers of abstraction and structured methods to keep that complexity manageable. In essence, it trades a limited capacity to manage complexity for the ability to finish far more complex tasks.

Technological evolution is, at its core, a move up the abstraction stack. Humans gave up low-level manual arithmetic so they could leap to high-level abstract orchestration.

As agents move further up the abstraction stack, the denominator of the TAM changes too.

Last time I did the math per person: long term, monthly AI spend per person could reach 10% of a white-collar salary, about $1,000. That’s the productivity argument, with opex as the denominator.

As agents keep moving up the abstraction stack, the denominator may stop being “agents per person” and become “agents per organization.” At that point agents decouple from headcount and wages, and they could even far outnumber the carbon-based population. It’s really the same story as unbounded tasks.

5. As AI makes information easier to access, insight easier to come by, and intent easier to turn into action, these gains combine to open markets an order of magnitude larger at each stage

Take the long view of how tokens have evolved, and there have been three stages so far.

Stage one, the chatbot, slashed the barrier to the flow of information.

Stage two, reasoning, slashed the barrier to insight: connecting the dots, knowing what to do in a complex situation, and spotting the right path among countless combinations.

Stage three, the agent, is slashing the barrier between intent and action.

Each stage opened up use cases an order of magnitude larger than the stage before, and the combinations between them will keep multiplying: lower barriers to acting on intent, combined with lower barriers to insight, open up even more room to grow.

Historically, every big drop in the information barrier set off a technological revolution. Drops in the barrier to acting on intent have too. But this is the first time in human history that the barrier is falling dramatically for intent that takes insight and intelligence to carry out, while the information barrier keeps dropping as well. It is bound to be an extraordinarily fast-growing technological revolution.

Every internet-era business boils down to one formula: a lower barrier to information flow, multiplied by any industry. Multiply it by human relationships and you get social networks; by transactions and you get e-commerce. The AI-era formula multiplies three forces by any industry: information that’s easier to access and share, insight that’s easier to come by, and intent that’s easier to turn into action.

Where these forces converge may be a fourth stage: the Proxy Economy. Whether the consumer is carbon-based or silicon-based, and whether it’s B2C or B2B, the potential is enormous.

The more capable the model, the more barriers an AI proxy can get past, and the more complex the work it can take on. The Proxy Economy still has a major threshold to cross. Over time it will become deeply integrated with social networks, e-commerce and every other field, moving toward agent-to-agent interaction. It may take one or two more leaps to get there. This isn’t a near-term story; on a longer horizon, it may be what keeps token growth exponential for a long time.

---

Demand beyond tokens

Finally, there’s a piece of compute demand that’s easy to miss: training.

The more profitable inference becomes, the larger the share of compute that can go to training. Take Anthropic: it will have roughly 10GW of compute in 2027. At a year-end ARR of $300B, about 4GW of inference capacity is enough, leaving 6GW that can go largely to training. In the first half of this year, when compute was tightest, inference may have accounted for two-thirds of capacity; next year, that could fall to just 40%. As RSI emerges, the returns increasingly justify the investment in training.

There is also training that doesn’t have to pay for itself through token revenue at all. Meta has no direct token revenue but can earn it back indirectly through products built on Muse. Google’s models, woven into its own ecosystem, generate large revenue outside of tokens. Even if they only make it into the second tier, they can’t walk away from training, and to keep up with the frontier, they can’t let the training-compute gap get too wide.

So AI infra demand isn’t just token demand. Starting next year, it’s training demand too.

VII. How this could play out: token spending may grow in jumps. Expecting a steady pace is unrealistic, and slowdowns may be perfectly normal

“Nothing picks up the baton after coding, so AI growth is doomed” is linear thinking

AI’s growth, and token demand in particular, may come in jumps rather than along a smooth line. New business models, new technological paradigms, or gradual improvements that finally unlock a fundamentally new capability can all trigger the next jump. “Nothing picks up the baton after coding” is a straight-line extrapolation of a process that moves in jumps.

Here’s why: capability improves continuously, but the payoff from token spending arrives in jumps.

1. Returns drive spending. AI is a game where you have to prove your return every moment of every day. What decides whether spending can keep doubling is the marginal return. A positive return on today’s token usage doesn’t mean twice the usage will earn the same return.

One study using OpenRouter and Azure data put the short-run price elasticity of token demand at roughly 1: cut the price 10% and usage rises about 10%, leaving total spend basically flat.

2. And the return on token spending is a threshold step function: cross the threshold and it jumps. However good L2 driver assistance gets, even L2.99, it’s still L2. The driver still can’t leave the seat, so it’s hard to get a jump in revenue. Only when it becomes L4 autonomy does the whole value pool open up at once.

Likewise, automating away the first 30–50% of the work has limited impact. A new paradigm only emerges once efficiency gains cross a threshold. Making employees less than 30% more efficient, for instance, usually doesn’t lower project costs. Only when the effort on a type of work drops by about 30% or more does it become practical to redeploy the freed-up staff to other projects.

Coding is the first domain to cross into L4, full autonomous takeover. That’s why token ARR exploded in the first half of 2026.

Back to token demand: whether total token spending can grow fast still comes down to returns, and most of the time, the growth in returns may not be all that exciting. You have to wait for the next jump.

Another conjecture: token spend for the top 1% and for median users is growing at roughly the same rate. Over the long run there may be a reversal: growth in median users’ token consumption could once again outpace the top 10%.

The logic is, power users account for a huge share of tokens, but not of GDP. The top 1% use 100x the median user’s tokens, but the revenue and GDP they generate are nowhere near 100x the median’s, and the gap between the top 10% and the median is nowhere near 10x either.

The ceiling on power users’ spending is their revenue, so sooner or later their spending growth has to converge to their revenue growth. Median users, on the other hand, still have plenty of room to spend more on tokens, and their spending can easily grow faster than their revenue.

---

Summary

Lay out the economics connecting tokens and semiconductors, and the picture is straightforward. Flagship tokens can generate $80B per GW a year. Spread the hardware cost over six years, and it comes to just $8B per GW a year. A tenfold gap.

Whether it’s open source or a kill line pushing token prices down, as long as costs fall in step, the token revenue a GW of hardware can generate stays the same, and so does the gross margin.

But you don’t hold on to these margins by standing still. You earn them through a relentless race to cut costs across the entire supply chain.

As for token demand and spending, the coming years will certainly bring more slowdowns lasting a few quarters. And there will always be people who mistake each slowdown for the end of the story.

We’re used to picturing the future as steady, constant-speed progress. Reality is more likely to unfold as a staircase, with sudden jumps from one level to the next.

At every step, someone will declare we’ve reached the peak/Plateau. But a step is a step precisely because there’s another one above it.

---

This piece has already run too long. I’ll go into the AI infra/semi civil war in the next one.