Overloaded & 529/503 Errors: Why LLM APIs Fail & How to Fix

"Overloaded," a 529 from Claude, a 503 "the model is overloaded" from Gemini, or a vague "chat completion api provider returned error" all point at the same thing: the provider is temporarily out of capacity for everyone, not you hitting your quota. It is a server-side condition you cannot fix by upgrading a plan — but you can engineer around it. This guide decodes the overloaded errors across Anthropic, Google Gemini and OpenAI, contrasts capacity 529/503 against your own 429 rate limit, and gives the three durable fixes: retry, exponential backoff with jitter, and failover so one provider's bad minute is not your app's outage.

What an 'overloaded' error actually means

It means the provider is temporarily out of capacity for everyone at once — a server-side condition, not you hitting your quota. Every major LLM API has its own way of saying it, but the meaning is identical:

• Anthropic Claude returns HTTP 529 with type overloaded_error — "The API is temporarily overloaded." It fires when the API sees high traffic across all users.
• Google Gemini returns HTTP 503 UNAVAILABLE — "The service may be temporarily overloaded or down," with the example cause "The service is temporarily running out of capacity."
• OpenAI returns HTTP 503 "Engine overloaded" — "Our servers are experiencing high traffic."

The critical property: because it is a capacity condition affecting all users regardless of pay tier, you cannot fix it by upgrading your plan. That is the single most useful thing to internalise — it separates overloaded from a quota error, which we decode next.

The cross-provider overloaded decoder

Same symptom, three status codes — here is every "overloaded" signal and its opposite (your quota) side by side. Built from each provider's official error reference:

Anthropic and OpenAI also return other 5xx codes worth handling the same way as overloaded — retry with backoff:

529/503 overloaded vs 429: the distinction that decides the fix

Overloaded is the provider's capacity; 429 is your account's limit — and confusing them sends you down the wrong fix. The two are opposites in who owns them:

• Capacity (529/503). The server is overloaded for all users. Not your quota, not your fault, and not fixable by upgrading a plan. The right move is to wait it out with backoff or route around it.
• Quota (429). Reflects your account's rate limits, quota or spend. On Gemini it is RESOURCE_EXHAUSTED (RPM/TPM/RPD/spend); on OpenAI it splits into "rate limit reached" and "out of credits / max monthly spend." This one is yours to manage — slow down, cache, or raise your tier.

One nuance that trips people on Anthropic: a sharp increase in your organisation's usage can produce a 429 via acceleration limits — so on Claude, a 429 is not purely a static quota signal. Still, the rule holds: 429 is account-side, 529/503 is provider-side. The account-side case has its own guide — see our how to fix LLM rate limits walkthrough and the Claude-specific rate-exceeded fixes.

Claude's overloaded error: HTTP 529 overloaded_error

A Claude "overloaded" error is HTTP 529 with type overloaded_error, meaning Anthropic's API is temporarily overloaded across all users. Anthropic's error reference states it plainly: 529 fires when the API experiences high traffic across all users — a capacity condition, not per-account quota. Contrast that with the 429 rate_limit_error, which means your account hit a rate limit (and, in rare cases, an acceleration limit on a usage spike).

Every Claude error is JSON with a stable shape — branch on error.type, never the message string:

{
  "type": "error",
  "error": {
    "type": "overloaded_error",
    "message": "The API is temporarily overloaded."
  }
}

Because 529 is not your quota, the fix is retry-and-route, not a plan upgrade. Anthropic's official SDKs already retry transient 5xx (and 429) twice by default with exponential backoff, honouring retry-after when present — so the first line of defence is often just leaving the SDK's retry behaviour on. Claude on our gateway is Claude Opus 4.8.

Gemini 'the model is overloaded': HTTP 503 UNAVAILABLE

"The model is overloaded" on Gemini is HTTP 503 with status UNAVAILABLE — the service is temporarily out of capacity, not your quota. Google's troubleshooting page describes 503 as "The service may be temporarily overloaded or down," with the example cause "The service is temporarily running out of capacity." Its recommended handling is deliberately simple and does not mention exponential backoff or a Retry-After header:

• Check the status page — look for a live incident on the Gemini API status page.
• Switch models temporarily — Google's own example is moving from Gemini 2.5 Pro to Gemini 2.5 Flash while the larger model is saturated.
• Wait and retry — give it a moment and try again.

Gemini's quota-side twin is 429 RESOURCE_EXHAUSTED, meaning you exceeded a rate limit (RPM/TPM/RPD/spend); the fix there is to verify you are within limits, retry after a short wait, reduce request rate or size, or request a limit increase. Note the "switch models" advice is exactly what a gateway automates — more on that below.

'chat completion api provider returned error' — decoded

This generic message is a wrapper: some client or gateway forwarded the upstream provider's failure without decoding it — so read the underlying status code, not the wrapper string. The wrapper tells you nothing on its own; the real diagnosis is one layer down:

If your client surfaces this string, log the wrapped HTTP status and provider error type and branch on those. A well-behaved LLM gateway or OpenAI-compatible endpoint should pass the upstream status through so your retry logic can distinguish an overloaded 503 from your own 429 — a wrapper that flattens everything to one opaque message is the thing to fix.

How to fix overloaded errors (with code)

Retry with exponential backoff and jitter — never a tight loop, which hammers an already-overloaded server — and treat 5xx differently from 429. The tactical and durable moves:

1. Exponential backoff + jitter on 529/503/500 — give the provider room to recover.
2. Honour retry-after when the provider sends it (Anthropic does; Gemini's docs do not document one for 503).
3. Switch models under load — Google's own advice (2.5 Pro to 2.5 Flash) generalises to any equivalent model.
4. Fail over across providers — the durable fix (see below).
5. Cap total attempts — bound retries so a sustained outage fails fast instead of hanging.

A provider-agnostic retry loop that backs off on overloaded and 5xx, but does not retry your own 429 the same way:

import time, random, httpx

OVERLOADED = {500, 502, 503, 529}          # capacity / 5xx — retry

def call(client, payload, tries=6):
    for i in range(tries):
        r = client.post("https://www.datallmlab.com/v1/chat/completions", json=payload)
        if r.status_code < 400:
            return r.json()
        if r.status_code in OVERLOADED:                 # overloaded — not your quota
            wait = float(r.headers.get("retry-after", 2 ** i))
            time.sleep(wait + random.random())          # backoff + jitter
            continue
        if r.status_code == 429:                       # your rate limit — respect retry-after
            time.sleep(float(r.headers.get("retry-after", 2 ** i)) + random.random())
            continue
        r.raise_for_status()                            # 400/401/403 — fix, do not retry
    raise RuntimeError("exhausted retries — fail over")

The official Anthropic and OpenAI SDKs already do most of this for you: both retry transient failures — connection errors, 429, and 5xx — twice by default with exponential backoff (Anthropic also honours retry-after). Tune the max-retries option rather than reimplementing; this loop is the explicit, cross-provider version for when you route to more than one API.

Failover turns an overloaded provider into a non-event

Backoff waits for one provider to recover; failover skips the wait by rerouting the same request to an equivalent model on another provider. An overloaded 529 or 503 is exactly the transient, not-your-fault error a gateway can absorb: on capacity failures it retries the request on a comparable model elsewhere, so the end user never sees "the model is overloaded" or "provider returned error." Because overloaded conditions are provider-specific and rarely correlated across vendors at the same instant, cross-provider failover is the highest-leverage resilience move — it converts a single provider's bad minute from an outage into an invisible reroute. The wiring is in our routing & failover guide; for picking resilient defaults, see the best LLM API roundup and, for agent workloads where an outage is most costly, the best LLM for AI agents.

FAQ