LLM API Retry Logs: Track Attempts Without Double-Counting Cost

When an LLM API call fails or streams incompletely, retry logic creates a new attempt, not a new request. Without distinguishing these in LLM API retry logs, cost reports eventually misstate spend. A vendor-neutral schema tracks usage and cost per attempt, combining them only when justified.

Separate one logical request from its individual attempts

A retry is not a second request from the user's point of view. It is your system trying again to satisfy the same request after an earlier attempt failed, errored, or returned an incomplete stream. If your logs only record "a call happened," you lose the distinction between three failed tries and one successful one, and any report built on that log will misstate both attempt counts and cost.

The fix is structural, not clever: give every logical request a stable ID that survives across retries, and give every individual call to the provider its own attempt ID underneath that request ID. One logical request can have many attempts. Each attempt is a separate HTTP call, with its own latency, its own response (or failure), and potentially its own provider-side request identifier.

This matters most when you're deciding what to bill a customer, what to report as API spend, or what to debug after an incident. Without the split, "how much did this request cost" and "how much did this attempt cost" collapse into the same unanswerable question.

Define a small vendor-neutral log record

You don't need a provider-specific schema to track LLM API retry logs correctly. A small, vendor-neutral record works across providers because it only commits to the relationships you actually need: which attempt belongs to which request, what state that attempt ended in, and how confident you are in the usage numbers attached to it.

A minimal event record needs these fields:

• logical_request_id: stable identifier for the request as understood by your application, unchanged across retries.
• attempt_id: unique identifier for this specific call to the provider.
• provider_request_id: the provider's own request identifier, when available. Not every provider returns one, and not every error path returns one even when the provider normally does.
• state: the outcome of this attempt (for example: succeeded, failed, timed_out, cancelled). Keep this a closed set of values your code actually handles.
• usage_source: where the usage numbers came from for this attempt, for example provider_reported, estimated, or unavailable.
• cost_state: whether a cost figure for this attempt is known, unknown, or explicitly not applicable (for example, a request cancelled before any billable work happened).

The two fields doing the most work here are usage_source and cost_state. They exist because an LLM API doesn't always tell you what you want to know after a failure. Decide early what partial output means for your app, since streaming usage reporting varies by provider and may be absent or cumulative during errors. If a stream drops halfway through, you may get no usage figure at all, or a figure that reflects only the tokens delivered before the drop. Your log record should let you say "unavailable" instead of quietly writing zero.

Work through a complete illustrative retry sequence

The following event log is a hypothetical example. No provider was called, and no real billing event occurred; every ID and value below is illustrative, built only to show how the schema in the previous section gets filled in across a retry sequence.

Hypothetical scenario: logical request req_8841 is sent to a provider. Attempt att_1 times out before any response. Attempt att_2 returns a server error mid-connection with no usage data. Attempt att_3 succeeds with output and a provider-reported usage figure.

In this hypothetical sequence, att_1 lacks a provider_request_id because the timeout preceded any response. att_2 has a provider request ID from the error response but no usage block. Only att_3 carries usage_source: provider_reported because it returned a complete usage object.

This hypothetical log does not indicate whether att_1 or att_2 were billed; that is provider-specific. It preserves a distinguishable record: three attempts for one logical request, one with known usage and two with unknown cost state.

Keep reported usage, estimates and missing cost information distinct

Treat usage_source and cost_state as different axes, not synonyms. A usage figure can be provider-reported, estimated by your own token counter, or simply unavailable. Separately, the cost derived from that usage can be known, unknown, or not applicable. Conflating these loses information you'll want later.

In this hypothetical example, att_1 and att_2 share usage_source: unavailable and cost_state: unknown for different reasons captured by state (timed_out versus failed), which suffices for later reconstruction without overloading the cost field.

Unknown is not zero. Writing cost: 0 for an unknown-cost attempt causes downstream sums to understate spend and obscures whether zero meant free or unmeasured. Recording cost_state: unknown preserves that distinction.

Reconcile the record before showing a request-level total

Before presenting a single total for hypothetical request req_8841, define what "total cost" means given the available attempt-level data.

In this hypothetical sequence, only att_3 has a known cost state. Aggregating by logical request yields: cost known for 1 of 3 attempts, with 2 attempts unknown. Presenting the known figure alone as the request cost misrepresents the actual information available.

Separate tracking is not a prohibition on aggregation. If a provider confirms failed or timed-out attempts are unbilled, aggregation is safe. Without that confirmation, treat unknown and zero as distinct. Reconcile known, estimated and missing values before deciding whether an aggregate total or per-attempt breakdown is more accurate.

Based on supplied answers and linked sources (2026-10).