MCP vs RAG: Different Layers, Same Context Window (A Decision Rule With Numbers)

Search MCP vs RAG and you get a wall of articles that pick a winner, usually implying RAG is legacy. That is a category error. RAG is an application-level strategy for deciding what text lands in the prompt. MCP is a JSON-RPC 2.0 client-server protocol for how an app connects to external systems at runtime — and its own documentation says it does not dictate how AI applications use LLMs or manage the provided context. By construction, MCP has no opinion about RAG. This piece replaces the versus framing with two real axes (when retrieval happens, and who decides), a numeric decision rule you can apply today, and the honest trade-off almost nobody prints: MCP has a context tax too.

Why MCP vs RAG is a category error

The comparison is popular because both things end up doing something that looks the same from the outside: text the model did not have before appears in the context window. But they sit at different altitudes, and the primary sources say so explicitly.

MCP's own architecture documentation describes the protocol as focusing solely on the protocol for context exchange and states that it does not dictate how AI applications use LLMs or manage the provided context. That single line settles the debate. MCP is plumbing — a JSON-RPC 2.0 data layer plus a transport layer. It carries bytes between a host application and a server. It has no view on chunking, embedding, reranking, prompt assembly, or whether you retrieve at all.

RAG is the opposite kind of thing. It is a design pattern at the application layer: split a corpus, index it, retrieve the top-k passages relevant to a query, and put them in the prompt before inference. It says nothing about how the retriever is reached over the network. You can implement RAG with a local library call, an HTTP API, a database driver — or an MCP tool.

So the two overlap in exactly one place: an MCP server can expose a retrieval tool. That is a real, shipping pattern, not a thought experiment. When that happens, you are doing both, and asking which one won makes as much sense as asking whether HTTP beat pagination.

The two axes that actually matter

Drop the versus. The questions worth arguing about are these two:

Axis 1 — when does retrieval happen? Classic RAG is pre-inference: a pipeline runs before or around the model call and hands it a finished block of text. The MCP/tool pattern is runtime: the model decides mid-conversation to call something and pulls data in as it goes. Anthropic's context-engineering guidance calls this second mode just in time context, and notes plainly that it costs latency: runtime exploration is slower than retrieving pre-computed data.

Axis 2 — who decides what enters the context? In RAG, your pipeline decides. A similarity function and a top-k cutoff pick the chunks; the model gets what it gets. That is deterministic and auditable, and it fails silently when the retriever misses. With tools, the model decides. That adapts to the actual question and handles multi-hop lookups, but it is non-deterministic, burns turns, and the model can simply choose wrong.

Every real trade-off in this space — freshness, latency, determinism, token cost, auth surface — falls out of those two axes. Which is why a comparison table without numbers on those axes adds nothing. Ours has them.

The decision rule, with numbers

Here is the part almost no MCP-vs-RAG article knows, straight from Anthropic's Contextual Retrieval post: if your knowledge base is smaller than 200,000 tokens (about 500 pages of material), you can skip RAG entirely, put the whole corpus in the prompt, and use prompt caching to make it fast and cheap — Anthropic cites prompt caching at more than 2x latency reduction and up to 90% cost reduction. Contextual Retrieval only becomes necessary as the knowledge base exceeds the context window.

Read that again if you are three sprints into a vector database migration. A very large fraction of internal-docs chatbots are under 500 pages. The correct architecture for them is a long prompt and a cache — no chunker, no embedding model, no reranker, no retrieval failure mode at all. Start from that null hypothesis and make the corpus prove it needs a pipeline.

The full branch:

• Corpus under ~200k tokens, static-ish, text-shaped → skip RAG. Stuff the context, cache the prefix. Zero retrieval bugs, because there is no retriever.
• Corpus bigger than the window, text-shaped, read-only → RAG, and budget for real engineering. Naive is not close to good (numbers below).
• Knowledge lives behind an API, changes between queries, or needs a write → a tool, plausibly over MCP. You cannot embed a live order status, and you certainly cannot embed a refund.
• Both true — a big static corpus and live systems → combine, which is what Anthropic actually recommends. See below.

Notice that MCP never appears as the alternative to RAG in that tree. It appears as the transport for the tool branch. You could serve that tool over a plain REST endpoint instead — which is a different and more honest comparison, covered in MCP vs API.

Side-by-side, with a numbers column

Table: DataLLM Lab. Token and tool-count figures are from Anthropic's Claude-specific tool-search documentation and describe Claude's behavior on Anthropic's API, not a universal property of the MCP protocol. Retrieval failure rates are relative reductions on one Anthropic internal eval published September 2024 and predate current models.

Both paths pay a context tax

Everyone writes that RAG bloats the context window. Almost nobody writes that MCP does too — and the number is not small.

Per Anthropic's tool search documentation: a typical multiserver setup (GitHub, Slack, Sentry, Grafana, and Splunk) can consume ~55k tokens in definitions before Claude does any work, and tool search typically reduces this by over 85 percent by loading only the 3–5 tools Claude needs. Separately: Claude's ability to pick the right tool degrades once you exceed 30–50 available tools. Aggregate a few MCP servers and you are at 200+ tools without trying.

That reframes the whole argument. RAG's token cost is variable — k chunks per turn, paid only when you retrieve. MCP's token cost is fixed and front-loaded — you pay for every tool definition on every turn whether the model uses them or not. Neither side gets to call the other bloated.

If you want the downstream consequence of filling that window carelessly, we covered it separately in context rot — long contexts do not degrade gracefully, and both taxes above are competing for the same budget you need for actual reasoning.

What MCP actually is in July 2026

Most articles freeze MCP at its November 2024 launch. Here is the current state, version-stamped, because this is a moving target.

Ownership. Anthropic introduced and open-sourced MCP on November 25, 2024. It is no longer Anthropic's protocol. On December 9, 2025 Anthropic donated MCP to the Linux Foundation's Agentic AI Foundation (AAIF), a directed fund co-founded by Anthropic, Block and OpenAI, with support from Google, Microsoft, AWS, Cloudflare and Bloomberg. Block's goose and OpenAI's AGENTS.md were donated alongside it. If a blog post calls MCP Anthropic's protocol in the present tense, that post has not been touched since launch — a useful staleness detector for everything else it claims.

Version. As of writing, the current spec is 2025-11-25. Versions are date strings marking the last date backwards-incompatible changes were made; they do not increment for compatible changes, and are marked Draft, Current or Final. Version negotiation happens at initialization. Do not trust an article citing 2025-06-18 or 2025-03-26 as current — 2025-06-18 appears as an example payload on the architecture docs page, which is a trap several writers have fallen into.

Imminent change. A release candidate dated 2026-07-28 is locked and described by the project as the largest revision of the protocol since launch: a stateless core that scales on ordinary HTTP infrastructure without sticky sessions; Extensions becoming first-class with reverse-DNS identification and independent versioning; Tasks graduating to an extension; MCP Apps adding server-rendered UIs via sandboxed iframes; six authorization hardening improvements including RFC 9207; a formal Active/Deprecated/Removed lifecycle with 12-month minimum transitions; and full JSON Schema 2020-12 for Tools. As of writing it is a release candidate, not shipped.

Architecture. An MCP host (the AI application — Claude Code, Claude Desktop, VS Code) creates one MCP client per server, each client holding a dedicated 1:1 connection. Two layers: a data layer (JSON-RPC 2.0 — lifecycle, primitives, notifications) and a transport layer (channels, framing, authorization). MCP is stateful and requires lifecycle management, though a subset can be made stateless over Streamable HTTP — which is the seam the 2026-07-28 RC pulls on.

Primitives. Servers expose three: Tools (executable functions the application can invoke), Resources (data sources providing contextual information), and Prompts (reusable templates structuring interactions). Each has discovery (*/list), retrieval (*/get), and in some cases execution (tools/call). The half that most write-ups omit is the client primitives: Sampling (sampling/createMessage — servers request an LLM completion through the client, so servers stay model-independent), Elicitation (elicitation/create — servers ask the user for information mid-operation), Roots (the client tells the server which filesystem directories to focus on), and Logging. Note that Roots are advisory only, file:// URIs only, and explicitly not a security boundary — the spec says servers SHOULD respect root boundaries, not MUST enforce. Do not describe them as sandboxing. There is also a cross-cutting utility, Tasks (Experimental) — durable execution wrappers for deferred result retrieval and status tracking — which is in motion, not a stable fourth primitive.

Transports. Exactly two standard transports: stdio and Streamable HTTP. Clients SHOULD support stdio whenever possible. Streamable HTTP requires a single endpoint supporting both POST and GET; SSE is optional within it for streaming. The old HTTP+SSE transport from 2024-11-05 is deprecated, kept only for backwards compatibility — not a co-equal third option. But the inverse error is also wrong: SSE was not removed, it lives inside Streamable HTTP. Practical details worth knowing: the optional MCP-Session-Id header assigned at init (client MUST echo it; a 404 means re-initialize); clients MUST send MCP-Protocol-Version on HTTP requests, with servers assuming 2025-03-26 if absent and returning 400 for an invalid one; resumability via SSE event ids and Last-Event-ID. And a security requirement people skip: Streamable HTTP servers MUST validate the Origin header on all incoming connections to prevent DNS rebinding attacks, SHOULD bind only to localhost when local, and SHOULD authenticate all connections.

Authorization — where most articles are now wrong. Authorization is OPTIONAL for MCP implementations. It applies to HTTP-based transports (which SHOULD conform); stdio implementations SHOULD NOT follow it and should instead retrieve credentials from the environment. So MCP requires OAuth is false. The stack, when used: OAuth 2.1 (still an IETF draft, draft-ietf-oauth-v2-1-13, not a ratified standard), RFC 8414, RFC 7591, RFC 9728, and OAuth Client ID Metadata Documents. The server is an OAuth 2.1 resource server; the client is an OAuth 2.1 client.

And the correction that is itself information gain: Dynamic Client Registration is no longer the primary path. As of spec 2025-11-25 there are three registration approaches in stated priority order — pre-registration, then Client ID Metadata Documents (CIMD, SHOULD support) where the client_id is an HTTPS URL pointing at a JSON metadata document, then DCR as a fallback MAY, described as included for backwards compatibility with earlier versions of the MCP authorization spec. Nearly every competing article still presents DCR as the modern MCP auth path. It is the legacy fallback. Hard requirements alongside it: servers MUST implement RFC 9728 Protected Resource Metadata; clients MUST use it for AS discovery; clients MUST implement RFC 8707 Resource Indicators (the resource parameter in both authorization and token requests, regardless of whether the AS supports it); clients MUST implement PKCE with S256; servers MUST validate token audience and MUST NOT pass client-supplied tokens through to upstream APIs.

Ecosystem. Ten official SDKs on a tier system: Tier 1 is TypeScript, Python, C# and Go; Tier 2 is Java and Rust; Tier 3 is Swift, Ruby, PHP and Kotlin. The MCP Registry — an open catalog and API for discovering public servers, launched September 8, 2025 — is still in preview, with no data durability guarantees and possible breaking changes or data resets before GA. And one more staleness detector: the current active reference servers are Everything, Fetch, Filesystem, Git, Memory, Sequential Thinking and Time. The launch-era list everyone still cites — Google Drive, Slack, GitHub, Postgres, Puppeteer — has been archived or handed to third parties (Slack is now maintained by Zencoder).

What RAG actually costs to do well

The case against RAG usually rests on it being fiddly. That is fair, and the numbers say how fiddly.

Anthropic's Contextual Retrieval work (published September 19, 2024) prepends chunk-specific context before embedding. Measured on top-20-chunk retrieval failure rate against an internal eval:

• Contextual Embeddings alone: 35% relative reduction in failure rate — 5.7% → 3.7%.
• Contextual Embeddings + Contextual BM25: 49% — 5.7% → 2.9%.
• Adding reranking on top: 67% — 5.7% → 1.9%.

Two readings. The optimistic one: retrieval quality is very much improvable, and the techniques are known. The one that matters more for architecture decisions: a naive pipeline sits at 5.7% failure against 1.9% for the engineered stack — roughly three times more silent misses. If someone tells you their RAG system is fine and they are running plain cosine similarity over default chunks, that is what fine costs. Caveats worth keeping: these are relative reductions in failure rate on one internal eval, not accuracy gains, not a guarantee for your corpus, and they predate current models.

That engineering burden is also why the model and embedding choices are not incidental — see the best LLMs for RAG and embedding model comparison if you have decided the pipeline is warranted. And the reason people reach for RAG in the first place — grounding — is worth reading against how to actually reduce hallucinations, because retrieval is one lever among several, not a cure.

The combine-both pattern

The hybrid is not a both-sides cop-out. It is a documented recommendation with a named cost, from Anthropic's context-engineering guidance:

• Today, many AI-native applications employ some form of embedding-based pre-inference time retrieval — RAG is not described as legacy.
• As the field transitions to more agentic approaches, we increasingly see teams augmenting these retrieval systems with just in time context strategies — augmenting, not replacing.
• The most effective agents might employ a hybrid strategy, retrieving some data up front for speed, and pursuing further autonomous exploration at its discretion.
• And the honest cost: There is a trade-off: runtime exploration is slower than retrieving pre-computed data.

Mechanically, the hybrid is simple. Pre-load the stable, predictable, high-hit-rate material before the turn starts — the pricing page, the schema, the policy doc — so the model is never waiting on a round-trip for things it always needs. Then expose the long tail as tools the model calls only when the question actually demands it. You pay the latency of a runtime fetch on the 20% of queries that need it, not the 100%.

The place the two layers genuinely touch is the obvious one: an MCP server can expose a search tool backed by a vector index. At that point retrieval is the tool, RAG is running inside MCP, and the versus framing collapses entirely. This is a common shape — managed remote servers that search documentation, and vector-database servers that expose semantic search with metadata filtering, both exist — and it is the cleanest proof that the layers compose. Which of the two you invoke is not an architecture decision; it is a routing decision the model makes per query. Structuring that well is the actual skill, and it is context engineering, not a protocol choice.

MCP is reinventing retrieval one layer up

Here is the observation that lands the thesis. If MCP were genuinely the successor to retrieval, its ecosystem would not be building retrieval systems for itself. It is.

Look at what the mitigations for tool sprawl actually are. Tool search: instead of putting all definitions in the prompt, index them and load only the 3–5 the model needs — that is top-k retrieval over tool definitions, cutting the ~55k baseline by over 85 percent. Anthropic's own thresholds for reaching for it: 10+ tools, definitions over 10k tokens, degrading selection accuracy, aggregated servers hitting 200+ tools, or a growing library. Below that — under 10 tools, under 100 tokens of definitions — plain tool calling is fine. Deferred loading is configured per-MCP-server rather than per tool definition.

Or code execution with MCP: present the tools as a filesystem of TypeScript files and let the agent read only what it needs. One Google Drive → Salesforce task went from 150,000 tokens to 2,000 tokens — a time and cost saving of 98.7%. Read that carefully, because the naive reading inverts it: that post is a critique of standard MCP tool loading, not a selling point for it. It is one illustrative example, not a benchmark or an average, and what it demonstrates is that loading everything up front was costing 75x more than loading on demand. The two problems it names are exactly RAG's problems: the token cost of definitions, and intermediate results repeatedly passing through the model.

Index a corpus, retrieve only the relevant slice, don't put the whole thing in the prompt. That is RAG. The corpus is just tool definitions instead of documents. MCP did not replace retrieval — it grew large enough to need retrieval of its own. Different layers, same problem, same solution.

A build checklist

Work these in order. Most teams skip the first two and regret it.

1. Count your tokens before you architect. Corpus under ~200k tokens (~500 pages)? Stop. Stuff the prompt, cache the prefix, ship. No vector DB.
2. Separate corpus from systems. List what is static text vs. what lives behind an API or changes between queries. The first is RAG's job; the second cannot be embedded and must be a tool.
3. Count your tools. Under 10, or under 100 tokens of definitions? Plain tool calling. Past 30–50 tools, expect Claude's selection accuracy to degrade — measure it before blaming the model.
4. Budget the window explicitly. Tool definitions + retrieved chunks + system prompt + history all draw on one budget. Write the numbers down. ~55k in definitions is not a rounding error.
5. If you build RAG, build it properly. Naive is 5.7% failure vs 1.9% engineered on Anthropic's eval. Contextual embeddings, BM25 hybrid, reranking. Budget for all three or accept 3x the misses.
6. Pre-load the predictable, tool the long tail. The hybrid Anthropic recommends. Pay runtime latency only on queries that need it.
7. Version-stamp your MCP assumptions. Pin to spec 2025-11-25 and watch the 2026-07-28 RC — stateless core, first-class Extensions, Tasks-as-extension, auth hardening. Anything unversioned in your docs will be wrong within a quarter.
8. Get the auth story right. stdio → credentials from the environment, no OAuth. HTTP → OAuth 2.1 draft with PRM (RFC 9728), Resource Indicators (RFC 8707), PKCE S256. Prefer pre-registration or CIMD; DCR is the backwards-compat fallback now. Validate Origin headers.
9. Re-run step 1 quarterly. Context windows and prompt caching keep moving the skip-RAG threshold. The pipeline you needed last year may be dead weight now.

FAQ

Written by Kevin Fan for DataLLM Lab. All MCP spec details are stamped to protocol version 2025-11-25 as of July 2026; the 2026-07-28 revision was a locked release candidate at time of writing.