Llama 4 Scout Review: 8/9 at $0.03 per 1,000 Tasks (The Cheapest and Fastest of the 45 Models We Have Run, and It Missed One)

We ran Llama 4 Scout through our executed coding benchmark on 2026-08-06. It scored 8/9, at a measured $0.03 per 1,000 tasks, in 1.5 s average, with 0 reasoning tokens per task. Both of those first two numbers are extremes: of the 45 models we have run as of 2026-08-06, none measured cheaper and none ran faster. The next cheapest is GPT-OSS-120B at $0.05, and two models tie for next fastest at 2.3 s: GPT-5.4 mini and Gemini 3 Flash Preview. But the headline is not the record. It is the task it missed: parse_csv_line, the fiddliest spec-following problem in our set and the single most-missed task across the whole dataset. DeepSeek V3.2 took all nine at $0.08 and 7.1 s. So the real question is whether 2.7x the money and 4.7x the wait is worth the ninth task, and that depends entirely on whether your work looks like parse_csv_line.

Llama 4 Scout is the cheapest thing we have ever put through this harness, and it is also the fastest. $0.03 per 1,000 completed tasks. 1.5 seconds a call. Both figures are the low mark across the 45 models we have run as of 2026-08-06. That is the kind of result that writes its own headline, so here is the sentence that should go above it instead: it got eight of nine.

Two superlatives and one miss is an awkward shape for a review, and most write-ups resolve the awkwardness by picking a side. We are not going to. The miss is not a rounding error and the price is not a gimmick, and which one dominates depends on a single question about your workload that we can state precisely by the end of this page.

What we measured on 2026-08-06

One run, nine executed Python tasks, temperature 0, one scored attempt each:

• Score: 8/9. The miss was parse_csv_line.
• Measured cost: $0.03 per 1,000 tasks, at list prices captured 2026-08-06.
• Mean latency: 1.5 s per task.
• Reasoning tokens: 0 per task. Scout does not think before it answers, and that is most of where the 1.5 s comes from.

Put those against the field as it stood on 2026-08-06. The next cheapest model we have run measured $0.05 — GPT-OSS-120B, also open-weight, also 8/9, also missing parse_csv_line. The next fastest averaged 2.3 s, and two models are tied there — GPT-5.4 mini, which scored 9/9 at $0.53, and Gemini 3 Flash Preview, again 8/9 with the same miss, at $0.36. Scout is 40% cheaper than GPT-OSS-120B and 35% faster than both of the 2.3 s models.

And the wide end of the spread, because it is genuinely hard to believe until you write it down: Gemini 2.5 Pro measured $28.58 on the identical nine tasks and scored 6/9. Divide $28.58 by $0.03 and you get 952.7, so Scout ran the same work for roughly one nine-hundred-fiftieth of the cost — we round that rather than print 953, because both inputs are rounded to the cent and a three-digit ratio built on two-decimal figures is false precision. It was also 16.9x faster: 1.5 s against 25.3 s. Gemini 2.5 Pro missed parse_csv_line too, plus roman_to_int and flatten.

The trade: one task for 2.7x the cost

Forget the records for a moment. The decision a reader actually faces is Scout against the cheapest model that took all nine.

DeepSeek V3.2 scored 9/9 at a measured $0.08 per 1,000 tasks in 7.1 s, priced 2026-07-30. Against Scout's 8/9 at $0.03 and 1.5 s, that is 2.7x the money and 4.7x the wall clock for one additional task.

Stated as absolutes rather than multiples, the trade shrinks dramatically: the gap is five cents per thousand tasks and 5.6 seconds per call. Nobody chooses an infrastructure model over five cents per thousand. So the multiple is the wrong frame for cost here — at this end of the field the entire cheap tier is free in any practical sense, and the money column stops being a tiebreaker somewhere around $0.10.

The latency gap is the one that survives contact with production. 1.5 s against 7.1 s is the difference between a completion that feels instant and one a user watches. If you are generating inline suggestions, autocompleting, classifying at the edge of a request cycle, or fanning out hundreds of small calls inside an agent step, 5.6 seconds per call multiplied by your call count is a real product constraint — the compounding we walk through in what AI agents actually cost.

So the honest framing is not cheap versus accurate. It is: you are buying one task with 5.6 seconds, not with five cents. Whether that is a good purchase depends on the ninth task, which is the next section.

The cheap end of the field, measured

The seven cheapest models we have run, ordered cheapest first — every model measuring under $0.40 per 1,000 tasks, with none skipped. Every number here is first-party: we ran each of these models on the same nine tasks through the same harness.

Two things fall out of that table that a leaderboard would hide.

First, the whole cheap tier is 12x wide and the score column barely moves. $0.03 to $0.36 is a twelvefold cost spread across seven models, and the range of outcomes inside it is 8/9 to 9/9. Gemini 3 Flash Preview costs 12x what Scout costs, is 0.8 s slower, and misses the same task. On this evidence, paying more within the cheap tier bought nothing at all.

Second, cost order and latency order are unrelated. The cheapest model is the fastest; the second cheapest is 8.6x slower than it; and the slowest row of all, DeepSeek V4 Flash at 14.5 s, sits in the middle of the price range. Reasoning tokens explain the latency column better than price does — the only two rows that emitted any reasoning tokens, GPT-OSS-120B at 198 per task and DeepSeek V4 Flash at 568, are also the two slowest at 12.9 s and 14.5 s, while the five models at 0 reasoning tokens all landed between 1.5 s and 7.1 s. If latency is your constraint, the reasoning-token column is the one to read, not the dollar column.

For where these six sit in the wider field, the AI coding ranking sorts the full set by each axis in turn, and the cheap coding model roundup works through the sub-$1 tier specifically.

Two open-weight models, same score, same miss

The single most useful comparison on this page is the one nobody markets: Llama 4 Scout against GPT-OSS-120B. Both are open-weight. Both scored 8/9. Both missed parse_csv_line and nothing else. They are, on this test, the same model in terms of what they got right.

They are not remotely the same model in terms of what they cost you to run:

• Cost: $0.03 against $0.05 per 1,000 tasks. Scout is 40% cheaper — though at these absolutes, two cents.
• Latency: 1.5 s against 12.9 s. Scout is 8.6x faster, and that is a difference you feel on every single call.
• Reasoning tokens: 0 against 198 per task.

The reasoning-token column is the mechanism, and it is worth saying plainly because it inverts a common assumption. GPT-OSS-120B spent 198 reasoning tokens per task thinking about these problems. Scout spent none. They arrived at the same eight answers and failed the same ninth one. The thinking bought nothing measurable here, and it cost 11.4 extra seconds per call plus the output-rate billing that reasoning tokens incur.

That is a narrow finding — nine bounded Python functions is exactly the regime where reasoning has least to add, and we would expect the picture to reverse on genuinely hard problems. But it is a real one, and it is the reason we publish the reasoning-token column at all. If you are choosing an open-weight model for high-volume bounded generation, a model that thinks is a model that waits. Local-deployment requirements for the other side of this comparison are in the GPT-OSS-120B requirements guide, and the broader open-weight field is in the best open-source LLM roundup.

parse_csv_line: the task money does not buy

parse_csv_line is the hardest spec-following task in our nine. It is not algorithmically difficult — it is a CSV field splitter. The difficulty is that the prose spec describes quoted fields, embedded delimiters inside quotes, and escaped quotes, and the hidden assertions check every one of those cases. There is no clever insight to find. There is only reading the specification carefully and implementing every clause of it, including the ones that look like edge cases.

It is the most-missed task in the entire dataset. Nine of the 45 models we have run as of 2026-08-06 missed it, against three misses for the next-worst task, flatten. Here is the full roster, ordered by measured cost:

That list spans $0.03 to $28.58. A model at 950x Scout's measured cost failed the same task. Frontier-priced models are on it and so are the two cheapest models we have run. If careful spec adherence were something you could buy with a bigger budget, this table would be sorted differently than it is.

Which turns the Scout decision into one concrete question. Does your work look like parse_csv_line? Concretely: does the model have to honour a long prose specification with several clauses that each only fire on unusual input — a parser, a validator, a format converter, a protocol handler, an escaping rule, anything where the failure case is silent and appears at 3am on one malformed row? If yes, spend the five cents and the 5.6 seconds. Take DeepSeek V3.2 at 9/9, or any of the 33 of 45 models that cleared all nine.

If your work is bounded generation against a clear, short spec — a transform, a small function, a schema-shaped extraction, a classification — then Scout's eight are the eight you need, and the ninth was never in your path. And in that regime nothing else we have run is faster or cheaper, as of 2026-08-06.

Open weights, and why our number says nothing about local speed

Llama 4 Scout is a Meta open-weight model. You can download it and serve it yourself, which is the property that separates it from most of the cheap tier and the reason its price behaves differently — you are paying a hosting provider for compute, not a lab for access to a model only they can run.

Our numbers are API-hosted, and that needs stating unambiguously: the 1.5 s we measured is a number about a hosted inference endpoint, not about Llama 4 Scout the weights. We sent requests over an OpenAI-compatible HTTP API to a provider running the model on their hardware. Run the same weights on your own GPUs and you will get an entirely different latency, determined by your hardware, batch size, quantisation, serving stack and network — and it could easily be slower or faster. Nothing on this page predicts it. Neither does the $0.03, which is a hosted list price multiplied by token counts and has no relationship to the cost of electricity and depreciation in your own rack.

Also worth being precise about the label: open weights is not open source. Meta publishes the weights under a licence with conditions; that is a different thing from an OSI-approved licence over the training pipeline and data. The distinction matters for what you are legally allowed to do with a fine-tune, and we take it apart in open weights vs open source. If you are evaluating Scout specifically because you want to self-host, the local coding model guide is the more relevant starting point, and none of its numbers are ours either.

The 1,310,720-token window we did not test

The headline capability of Llama 4 Scout is context. Per the OpenRouter model catalogue our model pages are built from, snapshot captured 2026-07-29, Scout's context length is 1,310,720 tokens — that is 1.25 Mi tokens, not a round 1.3M, and the exact figure is worth quoting because it is what the API will actually accept. The same catalogue entry describes Scout as a mixture-of-experts model activating 17B parameters of a 109B total, with text and image input. Those are catalogue facts sourced from Meta, not our measurements.

Two things follow. First, window size is a ceiling, not a performance guarantee — retrieval quality degrades unevenly as a window fills, the failure mode is quiet, and it is workload-specific enough that you have to measure it on your own documents. That is the problem we lay out in context rot. Second, a very large window on a very cheap model is an unusual combination and an attractive one for bulk document work, which is exactly why it deserves a real test rather than an inference from a coding score. We have not run that test.

The same disclaimer covers vision. Scout accepts image input per the catalogue; our harness is text-only and we sent no images. We have no data on it.

Who should run Llama 4 Scout

Run it for high-volume, latency-sensitive, bounded generation. 1.5 s and $0.03 per 1,000 tasks is the best combination of those two axes across the 45 models we have run as of 2026-08-06, and on eight of our nine tasks the output was correct. Inline suggestions, bulk transforms, classification, schema-shaped extraction, cheap first-pass drafts inside a pipeline that verifies them — this is the shape of work Scout is priced and clocked for.

Do not run it where a silent spec violation is expensive. Parsers, validators, format converters, anything with clauses that only fire on unusual input: it missed the one task in our set built exactly like that. Five cents per thousand tasks buys you DeepSeek V3.2 at 9/9. That is not a close call on price; it is a call about 5.6 seconds of latency.

Prefer it over GPT-OSS-120B for hosted API use on this evidence. Same score, same miss, 8.6x faster, 40% cheaper. The case for GPT-OSS-120B is about self-hosting economics and licence terms, not about our measurements.

Do not treat it as a frontier substitute. Nine short Python functions sit far below the level at which model tiers separate, so a near-tie here is evidence that the test did not reach the tiers, not that the tiers are equivalent. Everything Scout is actually differentiated for — the 1,310,720-token window, image input, self-hosting — we did not test.

How these numbers were produced

Nine executed Python tasks: two_sum, valid_parentheses, merge_intervals, roman_to_int, lcs_len, flatten, top_k_words, token_bucket, parse_csv_line.

The model receives a function signature and a prose spec. It never sees the assertions. The returned code runs against hidden assertions in an isolated python3 -I subprocess with a 12-second timeout. Every assertion passes or the task fails — no partial credit, no human grader, no LLM judge.

Temperature 0, max_tokens 4000, no effort or thinking-budget parameter set. One scored attempt per task. The harness retries only on an API error, never on a wrong answer, which is why Scout's miss stayed a miss. An empty response counts as an API failure and is dropped from the denominator rather than scored as wrong — a distinction that used to be a bug on our side, documented in the content-filter scoring bug.

Cost is derived, not invoiced. It is the token counts the API reported multiplied by that model's list price on a stated date. Llama 4 Scout ran on 2026-08-06 and is priced at 2026-08-06 rates; the other models in the tables carry their own dates in the date column, spanning 2026-07-17 to 2026-08-06. List prices move between captures, so an undated cost figure is not a fact — the habit we argue for in LLM price volatility. Recompute before you act on any of it.

Calls go through OpenRouter's OpenAI-compatible endpoint, deliberately not through the DataLLM Lab gateway. Nothing on this page depends on our infrastructure and you do not have to be our customer to reproduce it.

On the counting. Our original sweep was 13 models run in one sitting. Llama 4 Scout is among the models run later on the same harness under the same settings, which brings the total to 47 entries, of which 45 are usable and 33 scored 9/9. Where this page says 45 models, that is the combined usable set as of 2026-08-06. The core sweep was and remains 13.

Two entries are excluded from every comparison and are never quoted as scores. Claude Fable 5 returned empty responses with finish_reason=content_filter on four tasks after three retries each, leaving 4/5 — not a comparable result. Qwen3.5-397B-A17B returned empty on token_bucket with finish_reason=length: it exhausted our 4,000-token ceiling on reasoning without emitting an answer, leaving 8/8. That ceiling is our constraint, not a defect in the model, and neither entry belongs in a ranking.

What we did not measure

Not measured at all: long-context retrieval, multi-file refactoring, agentic or multi-turn tool use, any non-Python language, and vision. The harness is single-turn, API-hosted and Python only. It does not run agents and it does not use tools. Scout's 1,310,720-token window and its image input both sit in that list.

Nothing here is a claim about self-hosted Llama 4 Scout. Every latency and cost figure on this page describes a hosted API endpoint. Local throughput, local cost and local quality under quantisation are all unmeasured by us and not inferable from anything above.

Nine tasks is nine data points. An 8/9 is one observation on one draw, not a claimed failure rate. We did not run it twice, we did not vary the prompts, and we did not buy any model a retry.

A known scoring artefact: the 4,000-token ceiling can truncate a very verbose model mid-answer, and a truncated answer scores as a miss. That penalises verbosity, which is a real production cost but is not the same thing as being wrong. Scout emitted 0 reasoning tokens and was nowhere near the ceiling, so it was not affected — its miss was a wrong answer, not a truncation.

Not tested, and never claimed as ours: Llama 4 Maverick, any Llama 3.x model, Ollama or any locally-run model, and any model not listed in our benchmark tables. If a first-party number for any of those appears anywhere on this site, it is an error.

FAQ