Best Model for Hermes Agent: What 45 Measured Models Say (and the One Thing We Did Not Test)

The best model for Hermes agent is DeepSeek V3.2 if you want the cheapest thing that still works, and GPT-5.4 mini if you want the fastest — but the number that should actually drive your choice is one almost nobody publishes: reasoning tokens per call. Across the 45 models we have run on our executed coding harness, reasoning spend ranges from 0 to 2,671 tokens per call, and reasoning bills at the output rate. Hermes is a multi-turn agent, so that per-call figure gets multiplied by every turn in the session. Same task, same score, and a 184x gap in projected spend between two models that both went 9/9. Up front: we have not run Hermes Agent itself. We tested models, not the framework, and the section on limits says exactly what that costs you.

Hermes Agent does not ship with a model. Picking one is your job, and the setup flow says so in as many words — hermes-agent.org walks you through Nous Portal, OpenRouter, a custom OpenAI-compatible endpoint or a local vLLM server and then tells you to choose your model (read 2026-08-08). That single choice is the difference between an agent that costs pennies a day and one that costs hundreds.

The variable nobody prices for you is reasoning tokens. On our nine executed Python tasks, DeepSeek V3.2 emitted 0 reasoning tokens per call and Gemini 2.5 Pro emitted 2,671. Reasoning tokens bill at the output rate. In a single-turn API call that is an annoyance. In a Hermes session that fires twenty tool calls, it is the whole invoice.

This page gives you a tiered recommendation backed by measured numbers, then tells you plainly where our evidence stops.

What Hermes Agent is, and who says so

Everything in this section is third-party. We have not run Hermes.

Hermes Agent is an open-source, self-hosted AI agent framework released by Nous Research in February 2026, MIT-licensed (hermes-agent.org/about, read 2026-08-08). It is not an IDE copilot and it is not a chatbot wrapper. You run it on your own hardware or your own cloud box, point it at the LLM of your choice, and it executes tasks end to end.

What the project documents it doing, per its own site and docs read 2026-08-08:

• Executes tasks, searches, writes and runs code, and manages files. Code execution has four backends — local terminal, Docker, SSH to a remote host, and Modal or Singularity for cloud and HPC. Web capability covers search, page extraction and full browser automation.
• Connects to messaging platforms including Telegram, Discord, Slack, WhatsApp and Signal, plus email and a CLI.
• Persistent memory across sessions. The site's phrasing: it remembers everything across restarts, so projects, preferences and context are always there.
• A built-in cron scheduler with delivery to any connected platform — daily reports, nightly backups, weekly audits, running unattended.
• A self-improvement loop. The skills documentation (hermes-agent.nousresearch.com/docs/user-guide/features/skills) describes the agent creating a skill for itself after completing a complex task of five or more tool calls, and also after recovering from an error or a user correction. It writes the procedure and its pitfalls into a reusable skill document via a skill_manage tool. Those skills follow the agentskills.io open standard, the same format we walk through in the agent skills marketplace.
• All data stays local — no telemetry, no cloud lock-in, per the project's own claim.

On adoption: multiple outlets reported on 2026-05-10 that Hermes Agent overtook OpenClaw to become the most-used agent on OpenRouter's global daily rankings, processing over 224 billion tokens in a single day against OpenClaw's 186 billion, and ranking #1 across productivity, coding agents, personal agents and CLI agents (MarkTechPost, 2026-05-10; TechTimes, 2026-05-15). OpenRouter ranks by paid token throughput, so that is usage, not a benchmark. If you are weighing the two, we wrote up the incumbent separately in OpenClaw explained.

Why the model choice is yours — and why it sets the bill

Two properties of Hermes make model selection unusually consequential.

First, it is model-agnostic by design. The framework is deliberately not tied to Nous's own models; you point it at any compatible endpoint. That is a genuine strength — it is the same seam that makes a gateway useful — but it means the framework will happily run something that costs 357x the cheapest option in our set (Gemini 2.5 Pro at $28.58 per 1,000 tasks against DeepSeek V3.2 at $0.08, and the expensive one scored worse), and nothing in the install will warn you.

Second, it is multi-turn. A copilot answers once. A Hermes task plans, calls a tool, reads the result, calls another, and repeats. Its own documentation treats five or more tool calls as the threshold for a task being complex enough to be worth remembering — which tells you five-plus is routine, not exceptional. Every one of those turns is a separate billed call.

So per-call cost multiplies by turns. And the largest, least visible input to per-call cost is reasoning tokens, because reasoning tokens bill at the output rate — the expensive rate — and they do not appear in the answer you read. A model can look cheap on a rate card and spend 2,000 invisible tokens thinking before every sentence.

We have that number for 45 models on one harness. That is the reason this page exists.

What to run: four tiers, from measured numbers

Our harness is nine executed Python tasks, scored by running the returned code against hidden assertions. 33 of the 45 usable models scored 9/9, so correctness is not the separator at this level — cost, latency and reasoning spend are. Here is the recommendation, in the order most people should read it.

Cheapest capable default: DeepSeek V3.2. 9/9, $0.08 per 1,000 tasks, 7.1 s average, 0 reasoning tokens, priced 2026-07-30. Of the 45 models we have run as of 2026-08-06, it is the cheapest that cleared all nine tasks; the next cheapest clean scores are Qwen3 Coder Next and DeepSeek Chat, both at $0.10. If you are running Hermes as a personal daemon on a cron schedule and latency is irrelevant, start here and only move if it fails you.

Fastest capable: GPT-5.4 mini. 9/9, $0.53 per 1,000 tasks, 2.3 s average, 0 reasoning tokens, priced 2026-07-30. It is the fastest of the 45 models we have run as of 2026-08-06 that scored 9/9. Latency matters more in an agent than in a chat box, because you pay it once per turn: 2.3 s versus 7.1 s over twenty turns is 46 s versus 142 s of wall clock on the same session. When Hermes is answering you in Telegram and you are waiting, that gap is the product. Details in our GPT-5.4 mini review.

If you want a frontier model in the loop: Claude Haiku 4.5 or Claude Sonnet 5. Haiku 4.5 measured 9/9 at $0.94 and 3.7 s with 0 reasoning tokens (priced 2026-07-29). Sonnet 5 measured 9/9 at $1.67 and 7.2 s, also 0 reasoning tokens (priced 2026-07-17). Both are cheap enough to survive a twenty-turn loop and neither burns invisible thinking tokens on the way. See the Haiku 4.5 review for the fuller picture.

Below is every model that scored 9/9 with zero reasoning tokens — 11 of the 45. This is the shortlist a multi-turn agent should be picking from, because the invisible cost multiplier is switched off on all of them.

Table: DataLLM Lab. Score, latency and reasoning tokens are measured on our nine executed Python tasks. Cost is the measured token counts multiplied by each model's list price on the date shown, not an invoice. The score column is not marked because every row is identical at 9/9, and the reasoning column is not marked because every row is 0 — that is the point of the table.

Fourteen models in the full set emitted 0 reasoning tokens; three of those (Llama 4 Scout, Gemini 3 Flash Preview, Claude Opus 4.7) scored 8/9 and are therefore not on this list. A miss inside an agent loop is worse than a miss in a one-shot call, because the agent will act on it.

The reasoning-token spread, drawn to scale

The chart is the argument. It is one bar per model, drawn on a single scale, showing reasoning tokens emitted per call on identical prompts.

Four of these models thought for zero tokens and still got every task right. One thought for 2,671 tokens per call and got three of nine wrong. Reasoning spend did not buy correctness on this workload. That is a statement about nine bounded Python functions, not about reasoning in general — but nine bounded Python functions are a decent proxy for the small, well-specified sub-steps an agent actually issues inside a loop.

What not to put in a 20-turn loop

Three of the models on this page are specifically bad choices for a multi-turn agent, and the reason is the same in each case: high reasoning spend, billed at the output rate, on every single turn. They are not the only offenders in the 45 — Mimo v2.5 emitted 1,402 reasoning tokens per call, the second-highest figure in the whole set, took 26.0 s and scored 7/9 ($0.44, priced 2026-07-30) — but these three are the ones people actually reach for.

• Gemini 2.5 Pro — $28.58 per 1,000 tasks, 2,671 reasoning tokens per call, 25.3 s, and it scored 6/9 (missed roman_to_int, flatten, parse_csv_line; priced 2026-07-30). Of the 45 models we have run as of 2026-08-06 it is the most expensive, and one of the slowest — only Mimo v2.5 Pro at 26.1 s, Mimo v2.5 at 26.0 s and Qwen3.7 Max at 25.8 s took longer. It is also the lowest score anywhere on this page. Inside Hermes that combination is the worst case: you pay the most, wait the longest, and are most likely to have the agent act on a wrong intermediate result. We compare it against the Anthropic ladder in Gemini 2.5 Pro vs Claude.
• Gemini 3.1 Pro — $14.70 per 1,000 tasks, 1,094 reasoning tokens, 10.8 s, 9/9 (priced 2026-07-29). This one is capable. It is also 184x DeepSeek V3.2's measured cost for the same score. See the Gemini 3.1 Pro review.
• GLM-5.1 — $4.35 per 1,000 tasks, 1,327 reasoning tokens, 23.6 s, 9/9 (priced 2026-07-30). The absolute dollars look survivable until you notice the 23.6 s. Twenty turns of that is roughly eight minutes of wall clock before Hermes says anything back. Our GLM-5 writeup covers the family.

None of these are bad models. They are bad loop models, which is a different claim. A reasoning model that spends 1,300 tokens deciding how to answer is doing something valuable when the question is hard and you ask it once. When an agent asks it twenty easy questions in a row, you pay for that deliberation twenty times and get nothing extra.

The 20-turn projection

Table: DataLLM Lab. The first four data columns are measured on our nine executed Python tasks at each model's list price on its stated date (2026-07-17 to 2026-07-30; see the tables above). The last two columns are arithmetic on those measurements, not observations. We did not run a 20-turn agent.

The headline comparison is the two 9/9 rows at each end: DeepSeek V3.2 at a projected $1.60 against Gemini 3.1 Pro at $294.00 — 184x, for the same score on the same tasks. Both got everything right. One of them thought for 1,094 tokens per call before doing so, and you pay for that thinking at the output rate, twenty times per session.

Context growth makes it worse, not better, and asymmetrically. A model that resends a long history and re-derives its reasoning each turn compounds on both axes. That mechanism — and how to shrink the resent context — is the subject of what a running agent actually bills and cutting token costs in coding agents. We are not re-deriving it here.

For your own token mix, the cost calculator does the arithmetic against current rates.

How these numbers were produced

Nine executed Python tasks: two_sum, valid_parentheses, merge_intervals, roman_to_int, lcs_len, flatten, top_k_words, token_bucket, parse_csv_line.

The model gets a function signature and a prose spec. It never sees the assertions. Returned code runs against hidden assertions in an isolated python3 -I subprocess with a 12-second timeout. Every assertion passes or the task fails — no partial credit, no human grader, no LLM judge.

Temperature 0, max_tokens 4000, no effort or thinking-budget parameter set. One scored attempt per task, retried only on an API error, never on a wrong answer. An empty response counts as an API failure and is dropped from the denominator rather than scored as wrong — a rule we had to add after finding the opposite behaviour, written up in the content-filter scoring bug.

Calls go through OpenRouter's OpenAI-compatible endpoint, deliberately not through the DataLLM Lab gateway. Nothing here depends on our infrastructure.

Cost is derived, not invoiced. It is the token counts the API reported multiplied by that model's list price on a stated date. Every price date printed on this page falls between 2026-07-17 and 2026-07-30, and it sits next to its figure. List prices move constantly — the habit we argue for in LLM price volatility is to recompute before you act. Never read these as real billed cost.

On the counting. The core sweep was 13 models run in one sitting, and that number stays true wherever you see it on this site. The rest ran later on the same harness under the same settings, bringing the set to 47 entries, of which 45 are usable and 2 are excluded. Where this page says 45 models or 33 of 45, that is the combined set.

The two excluded entries, and why they are not comparable scores. Claude Fable 5 returned 4/5 — four tasks came back empty with finish_reason=content_filter after three retries, so it never completed the set. Qwen3.5-397B-A17B returned 8/8 — token_bucket came back empty with finish_reason=length, meaning the model exhausted our 4,000-token ceiling on reasoning without emitting an answer. That ceiling is our constraint, not a model defect. Neither figure belongs in a table beside a 9/9, and we do not print them as one.

What we did not measure, starting with Hermes

We did not test Hermes Agent. We tested models. That distinction carries real weight, and here is exactly where it bites:

• The skill-writing loop changes the token profile in ways we cannot see. When Hermes writes itself a skill after a five-plus-call task, it spends tokens to produce the document and then saves tokens on every later run that reuses it. Our single-turn harness observes neither the spend nor the saving. A model that is expensive per call could plausibly amortise better if it writes better skills — we have no data either way, and anyone who claims to is guessing.
• Persistent memory changes what gets resent. Hermes carries context across restarts. How much of it lands in each prompt is a configuration and implementation question, and it directly sets your input-token bill. Our harness sends 9 fixed prompts with no history at all.
• A self-hosted agent's real cost depends on how much context it resends each turn — the single largest lever, and one we do not touch here. The agent cost page covers it properly.
• Agent-specific failure modes are not in our score. Tool-call formatting, loop detection, knowing when to stop, recovering from a bad intermediate result: a 9/9 on nine Python functions predicts none of it. The eight agent failure modes is the page for that, and best LLM for AI agents is the broader ranking.
• Running an agent on your own box has a security surface we did not test — shell access, browser automation, five messaging integrations. See agent security.

Not measured at all: long-context behaviour, multi-file refactoring, agentic or multi-turn tool use, any non-Python language, vision, and anything run locally. The harness is single-turn, API-hosted and Python only. If you plan to run Hermes against a local model through Ollama or vLLM, we have no first-party numbers for you — local inference has a completely different cost structure and none of our dollar figures transfer.

Nine tasks is nine data points. A 9/9 is a clean result on this set, not a claim about a distribution. We did not run it twice, we did not vary the prompts, and no model got a retry on a wrong answer.

One known artefact: the 4,000-token ceiling can truncate a very verbose model mid-answer, and a truncated answer scores as a miss. That penalises verbosity, which is a genuine production cost but is not the same thing as being wrong. It is also what removed Qwen3.5-397B-A17B from the set.

FAQ