One price fell. The full cost became harder to read.
Headline API rates are dramatically lower than they were at GPT-4's launch. But “AI got cheaper” compresses several different things into one sentence: model tier, input and output rates, reasoning effort, prompt length, and how many calls an application makes.
OpenAI's flagship input price fell 24-fold — then quadrupled
Selected launch prices within one provider show the arc clearly: a steep fall from GPT-4 to GPT-5, followed by a rebound as the flagship tier became more capable and more expensive per token.
Today's menu spans an eighteen-fold gap
Standard, uncached input prices from five current endpoints range from $0.14 to $2.50 per million tokens. They are not equivalent in capability, context, latency or output price — which is precisely why the cheapest list rate is not automatically the cheapest workload.
The reasoning tax
Reasoning models can generate internal tokens before returning an answer. Those tokens are billed at the output rate but are not shown to the user. The amount depends on the model, task and effort setting, so a low input rate alone cannot predict the final cost.
Turning up "thinking" costs far more than it earns back
In one 260-question ophthalmology benchmark, the same GPT-5 model answered at three reasoning-effort settings. More effort raised both token use and accuracy — but on this already high-scoring task, the cost moved much more.
Going from low to high effort multiplied mean reasoning tokens more than six-fold (172 → 1,087) and mean cost six-fold ($0.002 → $0.012) while accuracy rose 1.5 percentage points (95.0% → 96.5%). This is one closed, medical multiple-choice benchmark — not a universal return curve. Source: Antaki et al., arXiv:2508.09956, Supplemental Table 5.
Some models hide most of their bill in tokens you never read
Across eight models on the same 12,267-example validation set of tool-use tasks, the share of output tokens spent on internal reasoning ranged from zero to nearly 70%.
GPT-5-nano averaged 170 output tokens per query, including 118 reasoning tokens: 69% of its output tokens. Kimi-K2.5 and Qwen-3.5-9B reported none on this benchmark. Reasoning tokens are billed at the output rate; total cost also includes input tokens. Source: Agarwal et al., arXiv:2605.07112, Table 26.
The management problem
Lower per-token rates did not make cost management disappear. As AI moved into more products and workflows, almost every FinOps respondent said managing AI spend had entered their remit.
AI cost management went from minority task to standard practice
Share of FinOps respondents who said they manage AI spend as part of their work.
Source: FinOps Foundation, State of FinOps 2026. The overall survey had 1,192 respondents representing more than $83 billion in annual cloud spend; the 2026 AI-management item reports N=693.
The sticker price and the workload can move in opposite directions
The list rate can fall while a particular query becomes more expensive. Model selection, prompt length, output length and reasoning effort determine how many priced tokens a workload consumes.
The unit-price crash is real. It simply does not settle the budget question. A workload's cost depends on which model handles each request and how many input, visible-output and reasoning tokens it uses. In the Switchcraft tool-use study, routing queries among models cut inference cost 84% while matching the best single model's accuracy — evidence that architecture can matter as much as the menu price.
The complete arc
Three chapters, one market.
The launch-price crash was real.
Within OpenAI's flagship line, standard input pricing fell from $30 per million tokens for GPT-4 to $1.25 for GPT-5 — before rebounding to $5 for GPT-5.6 Sol.
Tokens are not all visible.
Reasoning can improve performance, but it also consumes billed output tokens. How much depends on the model, task and effort setting.
The bill became a systems problem.
Routing, context length, output verbosity and usage determine whether lower rates become lower costs. FinOps teams increasingly have to manage the difference.