What Is an MCP Gateway? (And Why It Is Not an LLM Gateway)

Two pieces of infrastructure in a modern agent stack have almost the same name and do completely different jobs. An LLM gateway routes requests to models. An MCP gateway routes tool calls to MCP servers. We operate the first kind, so this guide is unusually blunt about where the line sits — and about when you should not put a gateway in front of your MCP servers at all.

What an MCP gateway actually is

An MCP gateway — also called an MCP proxy or an MCP aggregator — is an infrastructure layer that sits between MCP hosts and clients on one side and many MCP servers on the other. Clients connect to one endpoint. The gateway does the fan-out.

Per Docker's own documentation for its gateway, the component is a centralized proxy between clients and servers that takes over server lifecycle, routing and authentication across all of them, so clients stop configuring each server individually.

The cross-cutting concerns a gateway typically centralizes:

• Auth and secrets. One place that holds tokens and injects them, instead of every client config carrying its own copy.
• Access control and tool filtering. Allow-lists that decide which tools from which servers a given agent or profile can even see.
• Observability. Logging and call tracing across every tool invocation, which is the only realistic way to answer what did the agent actually do after the fact.
• Rate limiting and retries, on gateways that implement them.
• A registry or catalog of available servers, so discovery is a lookup rather than a Slack message.

One correction worth making up front, because a lot of blog posts still get it wrong: MCP is not Anthropic's protocol any more. It was donated to the Agentic AI Foundation, a directed fund under the Linux Foundation, on 9 December 2025. Every MCP gateway you will read about is third-party, community or vendor infrastructure — none of them is a first-party protocol component.

The no-gateway baseline (and why it stops scaling)

Start with what you get for free. Plain MCP configuration means each host — Claude Desktop, your IDE, your production agent — holds a config file listing each server, its transport, and its credentials. For a single developer with a filesystem server and a Postgres server, this is genuinely fine. It is also how most people start, and it is what our roundup of the best MCP servers assumes.

The failure mode is combinatorial, not conceptual. Every client-to-server pair is its own wiring: its own credential copy, its own transport quirk, its own place to forget an update. Gateway vendors like to quote the arithmetic that five MCP servers across three agents means fifteen separate OAuth implementations, credential stores and audit trails. Treat that number as illustrative rather than measured — it comes from vendor marketing, and real stacks share more than the arithmetic implies — but the shape of the curve is right and it is the honest reason gateways exist.

MCP gateway vs LLM gateway: different layers, same stack

This is the part almost every explainer blurs, so here is the cleanest framing we have found, from Cequence's write-up on the distinction: an LLM proxy is concerned with sending requests to models and returning responses; an MCP gateway is concerned with what happens after that response — which server, which function, which arguments the agent may invoke.

Put differently:

• LLM gateway = reasoning layer. Provider selection, failover, cost tracking, response normalization across vendors. That is what an LLM gateway is for, and it is what we build.
• MCP gateway = action layer. Authentication to tools, permissioning of tools, routing of a decided tool call to the right server, and the audit record of it.

They compose. Your app calls the LLM gateway to think; the tool calls that come out of that thinking flow through the MCP gateway to act. Neither replaces the other, and a gateway that claims to do both is doing two jobs with two very different threat models.

We will say the uncomfortable part plainly, since it is the whole credibility of this article: DataLLM Lab is an LLM gateway, not an MCP gateway. We route requests to 300+ models through one OpenAI-compatible endpoint and we pass tool definitions and tool results through as part of the chat payload. We do not host, discover, or authenticate MCP servers. If you need MCP fan-out, run one of the projects below next to us.

Three-way comparison: no gateway vs MCP gateway vs LLM gateway

Most comparisons online pit MCP gateways against LLM gateways and skip the baseline entirely, which hides the real decision. Here is the version with all three columns.

Read the row you care about, not the whole grid. If your answer to when you need it is still the first column, the rest of this article is background reading.

Two real gateways worth knowing

Docker MCP Gateway

docker/mcp-gateway is an open-source Docker CLI plugin, MIT-licensed and written almost entirely in Go — roughly 1.5k GitHub stars and around 71 releases at the time of writing, both self-reported repo metrics that move constantly. You start it with docker mcp gateway run.

Its verified features: MCP servers run as Docker containers with minimal host privileges, tool allow-lists let you enable or disable specific tools per server via profiles, secrets and credentials are injected through Docker Desktop rather than pasted into config, and OAuth flows are built in. Container isolation is the real differentiator here — it is the most concrete answer available to the MCP security problem of running arbitrary third-party server code on a developer machine.

One naming trap: Docker uses MCP Gateway for two different things. There is the MIT open-source CLI plugin above, and there is an invite-only Docker AI Governance enterprise feature that adds logging and call tracing for visibility over AI tool activity. They share a name; they are not the same product, and the governance tier is not the open-source one.

IBM ContextForge

IBM/mcp-context-forge is Apache-2.0 licensed and considerably broader in scope: an AI gateway, registry and proxy that federates MCP servers, A2A agents and plain REST or gRPC APIs behind one unified endpoint. It exposes virtual servers — curated bundles of tools drawn from multiple backends — plus centralized discovery, guardrails, JWT, OAuth and Basic auth, rate limiting, retries and OpenTelemetry observability. It is led by IBM Distinguished Engineer Mihai Criveti and reports roughly 4.1k stars, 3,000+ commits, 7,000+ tests and v1.x status; again, self-reported metrics, not audited figures.

The practical split: Docker's project is the strongest answer if your problem is running servers safely on developer machines. ContextForge is the strongest answer if your problem is federating heterogeneous backends for a whole organization. Both assume you already understand remote MCP server transports and how MCP authentication works, because a gateway does not remove those concerns — it concentrates them.

Honest trade-offs

A gateway is not free architecture. Three costs, all real:

1. It is a new component you must operate. Run it, patch it, monitor it, keep it highly available. Gateway vendors themselves say production deployments should be clustered and multi-region with health checks and failover — the same operational bar as an API gateway or an identity provider. That is a nontrivial amount of work to take on for a tool router.

2. It is a new trust boundary and a single point of failure. Every tool call and every credential passes through it. If the gateway cluster goes down, all agent capabilities are severed at once — not degraded, severed. Compare that to direct config, where a broken server costs you exactly one tool.

3. It adds a hop. Tyk, a gateway vendor, argues the cost is small: a high-performance gateway written in Go typically adds single-digit milliseconds of overhead per request, which Tyk calls a negligible price for the security and control you get. Take that as a vendor claim, not an independent benchmark. The counter matters more: agents chain many tool calls per task, so per-hop latency compounds across the chain rather than being paid once. Measure it on your own traffic before you assume it disappears.

There is a fourth, softer cost: a gateway that filters tools is also a gateway that can silently hide a tool your agent needed. Allow-lists are a security win and a debugging tax. If you are still deciding whether MCP is the right integration shape at all, our comparison of MCP vs a plain API is the earlier question to settle.

A decision rule you can apply in one minute

The crispest formulation we have seen comes from gateway vendors arguing against their own product in the small case: a central gateway is not worth the hop if your agents live on one developer's laptop.

Expanded into something you can actually run:

• Skip the gateway if you have one or two servers, one host, and no compliance requirement to produce an audit trail. Direct config is less code, fewer failure modes, and easier to reason about.
• Consider container isolation only — Docker MCP Gateway with allow-lists, no federation layer — if your risk is untrusted third-party server code rather than sprawl. This is the most common real situation for individual developers and it does not require an enterprise gateway.
• Adopt a full gateway when at least two of these are true: more than one team consumes the same servers, credentials must not live in client configs, tool access must be scoped per agent, or someone will eventually ask for a complete record of what the agents did.
• Do not adopt one to solve a model problem. Multi-provider routing, failover and cost control are the LLM gateway's job. Putting them in the tool layer is how stacks end up with two half-gateways and no clear owner of either concern.

The mental model that keeps this straight: ask whether the thing you want to control happens before the model answers or after. Before is the model layer. After is the tool layer. Almost every architecture argument about gateways resolves once you place the concern on that timeline.

FAQ