Self-Hosted LLM Gateway vs Managed: The Honest Cost Model (2026)

Start with the conflict of interest: we sell a managed LLM gateway. If you self-host, we do not get paid. That is a real bias and you should read every number below through it. So here is the arrangement — every figure on this page is either a first-party measurement you can check against our published benchmark, or a modelled assumption we name explicitly so you can swap in your own. The conclusion is not a recommendation. It is a break-even formula: self-hosting wins when your monthly infrastructure plus engineer-time costs less than the margin a managed gateway takes on your provider spend. At a 5% margin and a modelled $10,000/month of running it yourself, that break-even sits near $200,000/month of provider spend. Below that, buying is cheaper on pure cost. Above it, building is. And cost is not the only axis — two of the four cases where self-hosting genuinely wins have nothing to do with money at all.

If you have already read what an LLM gateway is and how routing and failover work, you know what the layer does. This page answers the question those two skip: should you run it yourself, and at what point does that stop being a hobby and start being cheaper?

Our conflict of interest, in full

DataLLM Lab sells a managed gateway. Every reader who decides to self-host is a customer we do not get. That is the plainest possible statement of the bias in this article, and putting it in the first screen is the only reason the rest of it is worth reading.

Two things follow from that. First, we are not going to tell you self-hosting is hard and leave it there — the cases where it is correct are listed below in more detail than the cases where it is not. Second, where we do use our own data, it comes from a benchmark that deliberately does not run on our infrastructure. All of our published model numbers are produced by calling OpenRouter's OpenAI-compatible endpoint, not the DataLLM Lab gateway, precisely so that nothing in them depends on our product working well. You can reproduce them without being our customer. On a page where we are otherwise selling something, that distinction is worth stating twice.

What we cannot remove is the framing bias — which questions we chose to ask, which costs we chose to model. So every assumption in the cost model has a name and a number you can overwrite.

Where self-hosting genuinely wins

Four cases. In the first two, cost does not enter the argument at all; the third changes what the cost is, and only the fourth is a pure spreadsheet question.

1. A rule forbids a third party in the path. This is the strongest case and it is not negotiable by spreadsheet. If your contracts, your regulator, or your customers' data-processing agreements say that prompt content cannot transit a vendor you have not individually vetted and named, a managed gateway is a new sub-processor and the answer is no. Health records, legal discovery, defence work, and a lot of EU public-sector procurement land here. Some of it can be solved with a signed DPA and a residency commitment; some of it genuinely cannot, and no discount rate changes that.

2. Air-gapped or VPC-only deployment. If the system runs inside a network with no egress to the public internet — or egress only to a specific allowlisted set of provider endpoints through a controlled NAT — then a gateway that lives outside that boundary is architecturally impossible, not merely undesirable. The same applies when you are serving open-weight models on your own hardware and the "providers" you are routing between are your own inference clusters. At that point the router has to live where the models live.

3. You already have a platform team who will own it anyway. This is the case people undercount, and it flips the arithmetic hardest. If you already run Kubernetes, already have on-call, already have secret management, already have a metrics pipeline and log retention, then a gateway is one more small stateless service in an existing chassis. The marginal cost is not "build and operate a gateway" — it is a few hours a month on top of infrastructure that exists regardless. We model that case separately below, because it is the one where self-hosting starts winning at realistic volumes.

4. Very high volume, where a percentage margin compounds. A managed gateway that takes a percentage of provider spend costs you nothing meaningful at $1,000/month and real money at $300,000/month, while the engineering effort to run one yourself is roughly flat across that range. This is the pure cost case, and it is the one the model below is built to size.

What you take on when you self-host

An open-source LLM gateway is not a library you import. It is a service that sits on the critical path of every request your product makes, which means it inherits every operational obligation of a service on the critical path. The list below is what changes hands.

None of these is hard in isolation. That is exactly why the cost is easy to underestimate: the build is three weeks and the operation is forever. The item that surprises teams most often is provider API drift, because it arrives unscheduled and it arrives as an outage. The item that costs most in aggregate is on-call, because it is a tax on a person rather than a line on an invoice.

Two of these have their own write-ups worth reading before you commit: rate limiting and 429 handling, which is harder than a token bucket in one process, and LLM observability, which is the part teams postpone and then need urgently the first time a bill triples.

The cost model, with assumptions named

Everything in this section is modelled, not measured. We have not run a survey of what it costs teams to operate a self-hosted gateway, and we are not going to pretend a plausible number is an observed one. What follows is arithmetic on six assumptions. They are all listed, all overwritable, and the formula at the end works with whatever you substitute.

• A1 — Engineer cost. $200,000/year fully loaded (salary, payroll tax, benefits, equipment, software) divided by 2,080 hours = $96/hour. Adjust hard for your market; this is a US-coastal figure and it is roughly triple some others.
• A2 — Initial build. 120 engineer-hours (three focused weeks) to get an open-source gateway to production quality: HA, key management, accounting, alerts. Amortised over 24 months = 5 hours/month.
• A3 — Steady-state ops. 4 hours/month at low volume rising to 55 at high volume, covering upgrades, drift patches, capacity work and on-call load.
• A4 — Cost per request. $0.00167, taken from our measured figure of $1.67 per 1,000 tasks for Claude Sonnet 5 on our nine-task benchmark. Our tasks are short single-turn coding problems. If your requests carry long context or run agent loops, yours will be much higher — substitute it.
• A5 — Managed margin. 5% of provider spend. This is a modelling parameter, not a quote; gateway vendors vary widely and some charge a flat platform fee instead. Sensitivity to this number is the whole story, so we vary it below.
• A6 — Infrastructure. Two redundant app instances, a Redis for shared rate-limit state, a Postgres for usage accounting, a load balancer, and log retention that grows with request count.

Read the last two columns only — both exclude the provider token spend in column two, because you pay that either way. What is being compared is the cost of the layer, not the cost of the tokens. At every one of these four volumes, under these assumptions, buying is cheaper: 118x cheaper at 100,000 requests/month, 18x at 1M, 3.5x at 10M, and 1.2x at 100M. The ratio collapses steadily, which is the real signal — the gap is closing fast by the fourth row and it closes entirely just above it.

We wrote that paragraph knowing exactly how it reads coming from us. So here is the counterweight, and it is the more interesting result.

Where the crossover actually sits

The four-row table is a snapshot. The formula behind it is the part you should keep:

Self-hosting is cheaper when (monthly infrastructure + monthly engineer-time) is less than (margin × monthly provider spend).

Rearranged, the break-even provider spend is your monthly cost to run it yourself divided by the margin. Two variables, and the margin dominates. Using the $9,960/month self-hosted figure from the 100M-request row, rounded to $10,000 so the arithmetic stays legible:

The chart is the argument. At a 2% margin you need half a million dollars a month of provider spend before self-hosting pays for itself, which is a small number of companies. At 20% you need $50,000/month, which is a great many companies. Before you model anything else, find out what margin you are actually paying. If a vendor will not tell you, that is information too.

Three things move the break-even that are easy to miss:

• Flat platform fees behave differently from percentage margins. A fixed $500/month does not compound with volume at all, so it never produces the high-volume crossover a percentage does. It is worse at low volume and better at high volume — the exact opposite shape.
• Bring-your-own-key arrangements change what the margin applies to. If you pay providers directly and the gateway only charges for the routing layer, the percentage does not sit on your token spend at all. That is a materially different deal — see how BYOK works.
• Routing savings dwarf both. A router that moves the right requests to a cheaper model reduces the provider spend that the margin is calculated on. Cutting your model bill by half is usually worth more than eliminating a 5% margin entirely. That is the next section.

Why you want a router either way

This part is measured, and it is the same regardless of who operates the gateway.

We ran 13 models through an executed coding benchmark in a single sitting: nine Python tasks, temperature 0, a 4,000-token ceiling, and every answer scored by running the returned code against assertions the model never sees. No partial credit, no human grader, one scored attempt per task. 10 of those 13 models scored a perfect 9/9. Among the models that scored perfectly, measured cost per 1,000 tasks ran from $0.10 to $8.83 — an 88x spread across models that were indistinguishable on correctness for this workload.

That 88x is the value a routing layer exists to capture. It does not care who owns the layer. If you route the routine half of your traffic from the $8.83 model to the $0.10 model and your quality bar holds, you have cut your provider spend far more than any gateway margin could ever have cost you — which also means you have moved the break-even in the chart above, because the margin now applies to a smaller number.

The practical order of operations follows from that: get the routing right first, then decide who operates it. Teams that do it the other way round spend three weeks building a gateway and then send every request to the same expensive default model, which is the worst of both outcomes. The mechanics of picking the routes are in the routing and failover guide, and the full 13-model field is in the coding cost benchmark.

One honest limit on that 88x: our nine tasks are short, self-contained Python problems. They are a fair test of whether cost separates models that correctness does not, and a poor test of long-horizon agentic work, where the models plainly do still separate. The usual right answer is a mix, not a single cheap default.

What is measured and what is modelled

Splitting these apart is the point of publishing numbers at all.

Measured. The benchmark figures only. Nine executed Python tasks — two_sum, valid_parentheses, merge_intervals, roman_to_int, lcs_len, flatten, top_k_words, token_bucket, parse_csv_line — at temperature 0 with a 4,000-token ceiling, scored by executing the returned code against hidden asserts in an isolated subprocess with a 12-second timeout. Retries happen up to three times on an API error and never on a wrong answer. Cost is computed: reported token counts multiplied by that model's list price. It is a measured cost, not a vendor invoice.

Modelled. Every number in the cost model and the chart. A1 through A6 are assumptions we chose. We did not survey teams, did not time a real build, and did not observe anyone's on-call load. The arithmetic on top of those assumptions is correct; the assumptions themselves are judgement. Change any one of them and the conclusion can flip — which is why they are enumerated rather than buried.

Not measured at all. Our harness is single-turn. It says nothing about long-context reasoning, multi-file refactoring, agentic or multi-turn tool use, non-Python work, or vision. The 4,000-token ceiling can truncate a verbose model and score it as a miss. And the calls go through OpenRouter's OpenAI-compatible endpoint, deliberately not through the DataLLM Lab gateway, so none of the numbers depend on our infrastructure.

Never claimed as ours. Any Gemini other than 3.6 Flash, GPT-5-mini and nano, Ollama or any locally-run model, Grok 3, Grok 4, Grok 4.5, Claude Haiku 4.5, Claude Opus 4.7, Claude Sonnet 4.6. If a number for any of those appears on this site attributed to our benchmark, it is an error and we want to hear about it.

A decision checklist

No recommendation. Work through these in order and the answer usually falls out on its own.

• Is a third party in the request path allowed at all? Check the contract and the DPA, not your instinct. If the answer is no, stop here — you are self-hosting, and the cost model is irrelevant.
• Does the system have egress? Air-gapped or VPC-only with no route to an external gateway is the same terminal answer.
• Are you routing between hosted APIs, or between your own inference clusters? If it is your own clusters, the router lives with them.
• What margin are you actually paying today? Get the number. Everything downstream scales inversely with it, and a flat fee behaves nothing like a percentage.
• What is your monthly provider spend, and what will it be in twelve months? Break-even is a spend threshold, not a request-count threshold. Estimate it with a cost estimate rather than a guess.
• Do you already have on-call, secret management and a metrics pipeline? If yes, drop the build amortisation and most of the ops hours — this single answer moves the break-even more than any other.
• Who is the named owner when the gateway is down at 03:00? If you cannot name a person, you do not have a self-hosting plan; you have a prototype.
• Have you already captured the routing savings? Cutting model spend is usually worth several times the margin. Do that first, then re-run the break-even on the smaller number — see cutting API costs.
• Can you get your data out of a managed gateway? If you buy, check that usage records and configuration are exportable, so that this decision stays reversible.
• What is the security posture either way? Self-hosting removes a sub-processor and adds a service you must patch. Both are real. The threat model does not go away by choosing one.

FAQ