Tokens keep getting cheaper and AI bills keep going up

Here’s a puzzle. Epoch AI estimates that the cost of reaching a given level of performance has been falling about 47% every quarter, roughly 13× a year, since 2023 (Epoch AI). Yet AI bills, for companies and for the labs themselves, keep growing.

Both are true, for three reasons.

1. Nobody buys a fixed level of intelligence

The 13× figure measures the cheapest model that matches a performance level that already exists. But people move to the newest, best model, and the newest models hold their price much longer. The discount arrives only once a cheaper or open-weight model catches up. Epoch’s own caveat is that the estimate assumes users switch to the cheapest adequate model, which in practice they rarely do.

2. Agents eat tokens

A chat answer might be a few hundred tokens. An agent that reads a codebase, plans, calls tools, retries and checks its work can use orders of magnitude more for one task. Cheaper tokens times far more tokens per task is still a bigger bill.

3. Cheaper things get used for more

This one is old economics, the Jevons paradox: when coal engines became more efficient in the 19th century, Britain burned more coal, not less, because coal became worth using for more things. Cheaper inference does the same. Tasks nobody would have paid a model to do two years ago are now automatic.

What I take from it

Optimise for cost per useful task, not cost per token. A smaller model that solves a well-scoped task in one try often beats a frontier model wandering through twenty tool calls. And the cheapest token is still the one you didn’t need to send.