Everyone Tells Me Compute Is Scarce. My Bill Disagrees.

In May a widely shared post from Liz Thomas said the compute shortage was pushing prices up: “Average LLM token costs are now $2.12/mil tokens, +12% this week alone and +65% since end of Feb.”¹ Five months later the index that number came from sits below $1 per million tokens, down 54% from its late-May peak of $2.07. A separate index built from what businesses actually spend on OpenAI and Anthropic is down 48% from its own late-February peak. Usage keeps climbing. Spending has been flat for two consecutive months.² My bill has been going in the opposite direction.³

What “average token cost” actually measures

The May post was not wrong about its snapshot. On May 18 the index really did read around $2.12. The mistake was reading a level as a trend. The index in question is Silicon Data’s LLM Token Expenditure Index, and its own methodology page is unusually honest about what it is: a usage-weighted blend of posted prices and observed volume, which “can rise when usage shifts toward more expensive models” and “can fall when lower-cost models gain adoption.” It is “not a vendor price sheet” and “not a guarantee of any customer’s realized invoice.”⁴

That last clause matters, because the metric cannot see what businesses are actually doing. Since February, buyers have gotten smarter about spend: routing work to cheaper open-weight models, taking the price cuts that come with each new generation, and letting cached inputs absorb long agentic conversations at a fraction of the rate. None of that is visible in an average price per token. The metric averages a market whose participants changed their behavior, then gets quoted as if it measured prices. It does not. Price per token is a cherry-picked number, and it misses the trend it is used to describe: the people paying the bills are paying less.

So the reporting I kept seeing in the spring, shortage drives prices up, was true for one week and false for the year. The people quoting it in May did not have bad data. They had a story that fits scarcity better than it fits a falling bill.

The second post is a different kind of story

The other post arrived October 23, and it comes as a chain. Elon Musk complained on an earnings call that additional AI compute is “utterly and totally unavailable,” that he cannot even find H100s to rent. Gregory C. Allen of CSIS quote-posted the clip with: “If we had complete and total AI chip export controls.” Then Chris McGuire finished the thought: “It’s not just H200 sales to China that are exacerbating US AI compute shortages. It’s also that Chinese companies can legally rent these chips in exactly the way Elon describes. To increase AI compute for US firms, we should ban Chinese firms from remotely accessing AI chips.”⁵

It reads like analysis. It is worth knowing who is talking. McGuire is a Senior Fellow at the Council on Foreign Relations, and in January he wrote the standard hawkish critique of the new export rule, closing with “The most effective policy remains the simplest one: denying the export of all AI chips to China.”⁶ The October post is that same argument wearing a scarcity costume. The remote-access ban he is calling for is not a discovery either; Commerce has been drafting exactly that rule since the summer, on top of the January rule that already caps China-bound H200s at half of US volume, taxes them 25%, and bars them from Chinese-owned data centers abroad.⁷⁸ The post is a policy ask with a market panic as its premise.

And notice what the whole chain never states: who is funding it, and what the agenda is. A think tank essay does not put its funders on the label, and neither does the rival center making the same argument under a different acronym. I cannot tell you whose money sits behind either position, and that is the point. Anything touching China and trade arrives pre-loaded with an agenda, and none of these voices disclose theirs. The scarcity story is the instrument. The chip makers want scarcity because it justifies pricing power. The advocates want it because it justifies restriction. Nobody in the chain has an incentive to say the true thing, which is that my bill is falling.

Both things are true, and that is the inflection point

Compute is scarce upstream and cheap downstream at the same time. You cannot rent an H100 easily at the top of the market, and H100 rentals have already collapsed by roughly two-thirds from their 2024 peak.⁹ Meanwhile the price of a token keeps falling because buyers keep routing around the expensive ones. Scarcity at the fab and glut at the API endpoint are one cycle seen from two ends, and most reporting sells one end as the whole thing.

So: date every number, separate a level from a trend, separate an expenditure index from an invoice, and ask who is funding the voice (chapter 11). Usage is up, unit prices are down, business spend is flat, and the loudest voices have reasons to tell you the reverse. My bill does not have an incentive. It just goes down.

Sources