DataLLM LabGPT-5.4 vs GPT-5.5: Identical 9/9, and a 5.2x Cost Gap (Measured)
We put four OpenAI tiers through the same nine executed Python tasks on one harness — three of them run on 2026-07-29, the GPT-5.5 flagship in our earlier core sweep. Every one of them scored 9/9. Then the numbers underneath the score came apart: measured cost ran from $1.53 to $8.83 per 1,000 tasks — a 5.8x spread inside one vendor's own lineup — and mean latency ran from 3.6 s to 15.2 s. The practical version: on this workload GPT-5.4 did everything GPT-5.5 did, at 5.2x less and nearly 3x faster. The strange version: the cheapest tier of the four, GPT-5 mini, was also by far the slowest.
Four tiers from one vendor. One harness, nine tasks the models never saw the tests for, one scored attempt each. All four came back with a perfect score, so the score is not the story. The spread underneath it is.
Read this first: what nine Python functions can and cannot tell you
This measures nine short, self-contained Python functions. Each task is a function signature plus a prose spec. The model returns code. The code is executed against assertions it never sees. That is the entire scope of what follows.
It says nothing about long-context reasoning, multi-file refactoring, agentic or multi-turn tool use, or any language other than Python. It is single-turn. Those are exactly the places where a newer flagship tier is most likely to pull away from a cheaper one, and we have no measurement of any of them. If your work is a 200,000-token codebase or a twelve-step tool loop, this page is not the evidence you need.
We put that at the top because the finding below is strong enough to be misread. A 5.2x cost gap at an identical score is a real result on a real workload — and it is a result about short, well-specified coding tasks, not about the models in general. Fuller boundaries are in what we did not measure and on the methodology page.
The four tiers, side by side
Sorted by measured cost, cheapest first. Cost is token counts we measured multiplied by list price on the date given in each row.
| Tier | Score | Measured cost / 1k tasks | Mean latency | Reasoning tokens | List price in / out per 1M |
|---|---|---|---|---|---|
| GPT-5 mini | 9/9 | $1.53 · 2026-07-29 | 15.2 s | 555 | $0.25 / $2 |
| GPT-5.4 | 9/9 | $1.69 · 2026-07-29 | 3.6 s | 0 | $2.50 / $15 |
| GPT-5.6 Sol | 9/9 | $4.98 · 2026-07-29 | 6.6 s | 58 | $5 / $30 |
| GPT-5.5 | 9/9 | $8.83 · 2026-07-17 | 10.5 s | 176 | $5 / $30 |
Four rows, one score. The cost column spans 5.8x, the latency column spans 4.2x, and the reasoning-token column spans zero to 555. Nothing in the score column would have told you any of that.
GPT-5.4 against GPT-5.5: same result, 5.2x less
GPT-5.4 scored 9/9 at $1.69 per 1,000 tasks in 3.6 s. GPT-5.5 scored 9/9 at $8.83 per 1,000 tasks in 10.5 s. Costs priced 2026-07-29 and 2026-07-17 respectively. Same nine tasks, same prompts, same grader, same temperature, one scored attempt each.
That is 5.2x on cost and 2.9x on speed, with zero measured difference in output quality on this set. There is no task where GPT-5.5 succeeded and GPT-5.4 failed, because there is no task either of them failed.
GPT-5.4 is also fast in absolute terms, not just relative to its stablemate. At 3.6 s it is the second-fastest model of the 21 we have run on this harness, behind only Mistral Medium 3.5 at 2.9 s, with Claude Haiku 4.5 third at 3.7 s. It is the fastest OpenAI tier we have measured by a wide margin.
Project that forward and the gap stops being abstract. A steady thousand tasks a day of roughly this size, at these prices, works out to about $617 a year on GPT-5.4 against about $3,223 a year on GPT-5.5 — a projection from our measured per-task figures, not a bill anyone received. For output that scored identically.
The plain reading: if you default to the newest flagship for short, well-specified coding work, you are paying a large premium for no gain that this benchmark can detect. Our fuller read on the flagship itself is in the GPT-5.5 review, and the wider field is ranked four different ways in the AI coding ranking.
Half the sticker price, one fifth the bill
This is where the numbers get more interesting than the headline. GPT-5.4 lists at $2.50 / $15 per 1M tokens. GPT-5.5 lists at $5 / $30 — exactly double, on both input and output. So a rate-card comparison predicts GPT-5.4 will cost half as much.
It cost one fifth as much. The measured gap was 5.2x, not 2x. The rate card explains 2x of it; the remaining 2.6x came from token volume — GPT-5.5 emitted 176 reasoning tokens per task and GPT-5.4 emitted zero, and reasoning tokens bill at the output rate.
GPT-5.6 Sol isolates that effect cleanly, because it lists at the identical $5 / $30. Same rate card as GPT-5.5, no pricing difference at all, and yet $4.98 against $8.83 per 1,000 tasks — 1.8x apart on token volume alone. With the rate held constant, how much the model wrote was the only variable, and it moved the bill by nearly a factor of two.
| Pair | List output rate | Measured cost gap | What moved it |
|---|---|---|---|
| GPT-5.4 vs GPT-5.5 | 2.0x cheaper | 5.2x cheaper | 176 reasoning tokens vs 0 |
| GPT-5.6 Sol vs GPT-5.5 | identical · $30 | 1.8x cheaper | token volume only |
| GPT-5 mini vs GPT-5.5 | 15x cheaper | 5.8x cheaper | 555 reasoning tokens ate the discount |
| GPT-5 mini vs GPT-5.4 | 7.5x cheaper | 1.10x cheaper | 555 reasoning tokens vs 0 |
Every GPT-5.5 comparison in that table prices its measured token counts at 2026-07-17 and the other three tiers at 2026-07-29. GPT-5.5's list price was $5 / $30 on both dates, so the gaps are unaffected — the price dates section works through it.
Read the last row twice. GPT-5 mini's output tokens cost one seventh and a half of GPT-5.4's, and its finished bill for the same nine tasks came in 9% lower. Not 87% lower. Nine.
The general lesson, which holds well outside OpenAI's lineup: a per-token rate is a rate, not a price. What you pay is tokens burned multiplied by rate, and models differ on both factors at once. If you want that arithmetic on your own token mix, the cost calculator does it.
The cheapest tier was the slowest tier
GPT-5 mini is the cheapest of the four at $1.53 per 1,000 tasks, priced 2026-07-29. It is also, by a distance, the slowest: 15.2 s mean per task, against 3.6 s for GPT-5.4. It is slower than every full-size OpenAI tier we tested, including the flagship.
The cause is visible in the token counts. GPT-5 mini spent 555 reasoning tokens per task. GPT-5.4 spent zero. Across these four tiers the latency ordering is the exact inverse of the reasoning-token ordering — 0, 58, 176, 555 maps to 3.6 s, 6.6 s, 10.5 s, 15.2 s without a single inversion. On this set, the mini tier bought its cheap per-token rate by thinking longer, and thinking takes wall-clock time.
Now put the two numbers together. Choosing GPT-5 mini over GPT-5.4 saves $0.16 per 1,000 tasks and costs 11.6 extra seconds per task. Across a thousand tasks that is 11,600 seconds — about 3.2 hours of additional waiting — bought for sixteen cents. Roughly five cents an hour of latency. If a human is watching a spinner, or if the call sits inside a loop, that trade is not close.
This is the part no competing comparison seems to have: the cheap tier is not the fast tier, and on this workload it is not really the cheap tier either once you price the waiting. Cheap per token bought latency, not savings. The mini against nano comparison covers the small end of the lineup further, though note we have never run nano on this harness.
Where GPT-5.6 Sol lands
GPT-5.6 Sol scored 9/9 at $4.98 per 1,000 tasks in 6.6 s, priced 2026-07-29, on 58 reasoning tokens per task. Against GPT-5.5 that is the same score, 1.8x less money, and 1.6x faster — on an identical $5 / $30 rate card.
Against GPT-5.4 it is the same score for 2.9x more money and 1.8x more time. On this workload there is nothing it did that GPT-5.4 did not, and we would not pay the difference for nine Python functions. Whether it earns that gap on longer or harder work is a question this harness cannot answer; our separate read is in the GPT-5.6 Sol review and the head-to-head in GPT-5.6 against GPT-5.5.
The price dates, stated plainly
Three of these four cost figures were priced on 2026-07-29 — GPT-5 mini, GPT-5.4 and GPT-5.6 Sol, all run that same day. The GPT-5.5 figure comes from our core 13-model sweep and was priced on 2026-07-17.
That is a twelve-day gap, and it matters because list prices move: 49 of roughly 396 models in the catalog changed price in the twelve days to 2026-07-29. GPT-5.5 was not one of them. Its list price was $5 / $30 per 1M tokens on 2026-07-17 and $5 / $30 on 2026-07-29, so re-pricing its measured token counts at the later date produces the same $8.83, and the comparison holds.
We are spelling that out rather than quietly presenting four numbers as if they came from one moment. Every cost figure on this site is only true as of its pricing date. Recompute before you act on any of it. Current rates are on the GPT-5 API pricing page.
Which tier we would actually run
The measurements above are arithmetic. This section is a judgement, so the reasoning is stated and you can reject it.
Default for short coding tasks: GPT-5.4. 9/9, $1.69 per 1,000 tasks priced 2026-07-29, 3.6 s, zero reasoning tokens. It is the fastest OpenAI tier we have measured, it sits 16 cents per 1,000 tasks above the cheapest of the four, and the zero reasoning count means its cost does not inflate when a task turns awkward. That combination is why it wins here rather than the nominally cheaper mini.
If latency is genuinely free: GPT-5 mini. Batch jobs, overnight runs, anything with no human waiting. It scored the same 9/9 and it is 9% cheaper. Just be honest with yourself about whether the 15.2 s is really free.
GPT-5.5 and GPT-5.6 Sol: not on this evidence. Both scored 9/9. So did GPT-5.4, at 2.9x less than Sol and 5.2x less than GPT-5.5. That is not a statement that they are weak models — it is a statement that nine short Python functions are far below the level where their price starts earning anything, and if you are choosing them it should be on capabilities this harness never touched. Long-horizon agent work is one plausible place they separate.
If you are picking one ChatGPT-family model for general use, the coding benchmark is only one input; our best ChatGPT model guide weighs the rest.
How these numbers were produced
Nine executed Python tasks: two_sum, valid_parentheses, merge_intervals, roman_to_int, lcs_len, flatten, top_k_words, token_bucket, parse_csv_line.
The model receives a function signature and a prose spec. It never sees the assertions. The returned code runs against hidden assertions in an isolated python3 -I subprocess with a 12-second timeout. Every assertion passes or the task fails — no partial credit, no human grader, no LLM judge.
Temperature 0, max_tokens 4000. One scored attempt per task. The harness retries only on an API error and never on a wrong answer, which is why a miss stays a miss.
Cost is computed, not invoiced: the token counts the API reported, multiplied by that tier's list price on the stated date. It is a measured cost, not a bill from a vendor. Calls go through OpenRouter's OpenAI-compatible endpoint, deliberately not through the DataLLM Lab gateway, so nothing here depends on our own infrastructure and you do not have to be our customer to reproduce it.
Our core sweep was 13 models run in one sitting and GPT-5.5 is one of them, priced 2026-07-17. GPT-5.4, GPT-5.6 Sol and GPT-5 mini were run later on the same harness, on 2026-07-29. Across all 21 models we have now run, 18 scored 9/9, measured cost spans $0.10 to $14.70 per 1,000 tasks, and seven models emitted zero reasoning tokens — GPT-5.4 among them.
What we did not measure
Not measured at all: long-context reasoning, multi-file refactoring, agentic or multi-turn tool use, any non-Python language, and vision. The harness is single-turn. It does not run agents and does not use tools. These four tiers may separate sharply on exactly those axes, and we would not know.
A known scoring artefact: the 4,000-token ceiling can truncate a very verbose model mid-answer, and a truncated answer scores as a miss. None of these four hit it, but it is a real ceiling on the method.
Sample size: nine tasks, one scored attempt each, temperature 0. A perfect score on nine problems is evidence of competence on this kind of problem. It is not a claim about a hundred.
Cost caveat: list prices move. GPT-5.5's figure is priced 2026-07-17 and the other three 2026-07-29; GPT-5.5's rate did not change between those dates, but rates in general do. Recompute against current prices before acting.
Not tested, and never claimed as ours: GPT-5 nano, Ollama or any locally-run model, Grok 3, Grok 4, Grok 4.20, grok-build-0.1, VibeThinker, and any Gemini other than 3.6 Flash and 3.1 Pro. If a first-party number for any of those appears anywhere on this site, it is an error.
Test the tier below your default
One OpenAI-compatible endpoint, 300+ models, one key. Change the model id, rerun your own tasks, compare the bill — that is the only comparison that predicts your costs.
FAQ
Is GPT-5.4 better than GPT-5.5?
On our benchmark they are indistinguishable on quality — both scored 9/9 on the same nine executed Python tasks, with no task either one failed. GPT-5.4 was cheaper and faster: $1.69 per 1,000 tasks priced 2026-07-29 against $8.83 priced 2026-07-17, and 3.6 s against 10.5 s. So for short, well-specified coding work GPT-5.4 is the better buy on this evidence. For long context, multi-file refactoring or agentic tool use we have no measurement, and the flagship may well be better there.
Why is GPT-5 mini slower than the full-size tiers?
Because it spent 555 reasoning tokens per task while GPT-5.4 spent zero. Reasoning happens before the answer arrives, so it lands directly in wall-clock time. Across these four tiers the latency order is the exact inverse of the reasoning-token order: 0, 58, 176 and 555 tokens map to 3.6 s, 6.6 s, 10.5 s and 15.2 s. A cheap per-token rate does not imply a fast response.
GPT-5.4 lists at half GPT-5.5's price. Why is the measured gap 5.2x?
The rate card explains 2x of it. The other 2.6x is token volume. GPT-5.5 emitted 176 reasoning tokens per task and GPT-5.4 emitted zero, and reasoning tokens bill at the output rate. GPT-5.6 Sol makes the same point with the rate held constant: it lists at the identical $5 / $30 as GPT-5.5 and still came in 1.8x cheaper on measured cost, purely on how much it wrote.
The GPT-5.5 number is priced twelve days earlier. Does that break the comparison?
No, in this specific case. GPT-5.5's list price was $5 / $30 per 1M tokens on 2026-07-17 and unchanged at $5 / $30 on 2026-07-29, so re-pricing its measured token counts at the later date yields the same $8.83. We flag it because 49 of roughly 396 catalog models did change price in the twelve days to 2026-07-29, so date-matching is not something we can wave away in general.
Is the measured cost what I would be billed?
No. We take the token counts the API reported and multiply by list price on a stated date. It is a computed measured cost, not a vendor invoice. Your bill will differ with prompt length, caching, discounts, provider routing and any price change since. The ratios between tiers are the durable part; the absolute dollars are not.
Does a perfect 9/9 from all four tiers mean the benchmark is too easy?
For separating these models on correctness, yes — and that is the finding, not a defect. Nine short Python functions are now inside the competent range of every serious tier shipping in 2026, so the discrimination moved to cost and latency, where it spans 5.8x and 4.2x. The scoring itself is strict: code is executed against hidden assertions, all pass or the task fails, one scored attempt, no retries on a wrong answer. If you need a benchmark that separates these tiers on capability, you need harder or longer tasks than these.