LLM Function Calling (Tool Use): One Loop, Two Very Different API Shapes

Function calling is conceptually simple and operationally fiddly. The concept: the model does not run your code, it returns a structured request asking you to run it. The fiddly part: OpenAI and Anthropic implement that same idea with different field names, different message shapes, different parallel-call switches, and — the detail that eats the most debugging hours — different argument types. This guide maps the two APIs field by field, walks the loop end to end, names the failure modes, and shows how to score tool-calling quality across models instead of guessing.

The loop, precisely

Every function calling implementation is the same four beats, and the vendor docs agree on all of them. You send a request that includes a list of tool definitions. The model decides it needs one and, instead of answering, returns a structured call: a tool name plus arguments. Your code executes that function. You send the result back in a follow-up request containing the full prior turn plus the result. The model reads the result and produces the final answer — or asks for another tool.

The critical mental model: the model never runs anything. It emits a request. Your process is the executor, which means your process owns authentication, rate limits, validation, and the blast radius. Anthropic makes this explicit by distinguishing client tools, which run in your application and require you to return a result, from server tools such as web search, web fetch, and code execution that run on Anthropic infrastructure. Generic function calling is the client-tool case.

Anthropic signals a tool request with stop_reason: "tool_use" and one or more tool_use content blocks. OpenAI signals it with a tool_calls array on the assistant message. Same beat, different instrument.

OpenAI vs Anthropic: the field map

Vendor docs describe each API in isolation and never map field names across providers. That map is the single most useful artifact if you support more than one model, so here it is. Every row was checked against the official tool-use pages as of July 2026.

Read that table once and you have already avoided the four bugs that dominate cross-provider ports: renaming parameters to input_schema, unwrapping the nested function object, parsing arguments on one side and not the other, and hunting for a top-level parallel flag on Anthropic that does not exist because it lives inside tool_choice.

The two OpenAI surfaces most tutorials conflate

OpenAI now documents two distinct request surfaces, and a large share of blog examples silently mix them. Chat Completions uses the nested function wrapper, returns tool_calls on the assistant message, and takes results back as messages with role: "tool" carrying a tool_call_id. The newer Responses API flattens the tool definition — {type, name, description, parameters, strict} with no inner object — emits an output item of type: "function_call" with call_id, and takes results back as an input item of type: "function_call_output".

They are not interchangeable. If you copy a definition from a Responses example into a Chat Completions call, the API will reject it for a missing function object; go the other way and your flattened parser will never find the fields. Pick one surface per code path and label it in comments. If you are consolidating on the older, more universally supported shape, that is exactly what an OpenAI-compatible API layer gives you across providers.

Strict schemas and parallel calls

Two levers separate a demo from something you can page someone about.

Strict mode. Both providers support a strict: true flag that constrains generated arguments to conform to your schema. OpenAI documents real constraints on it: additionalProperties must be false, every property must appear in required, and only a subset of JSON Schema is supported. That is not a nuisance, it is the price of a hard guarantee — if a field is genuinely optional, model it as a nullable union rather than omitting it from required. Anthropic documents an equivalent strict tool-use option, set as a top-level property on the tool definition, with its own narrower supported subset of JSON Schema. Strict mode fixes shape, not judgment: a perfectly valid object can still contain a wrong city name. For the neighbouring problem of constraining ordinary responses, see structured output with Claude.

Parallel calls. Both providers enable parallel tool calls by default: one assistant turn can request several tools at once, which is a large latency win when the calls are independent (three product lookups) and a correctness hazard when they are not (create the order, then charge it). Turn it off with a top-level parallel_tool_calls: false on OpenAI, or by setting disable_parallel_tool_use: true inside the tool_choice object on Anthropic. Different mechanism, same intent.

One cost note that surprises people: on Anthropic, simply passing a tools parameter injects a tool-use system prompt that is billed as input tokens, and the size varies by model and by which tool_choice you use. Vendor-reported and version-specific, so do not hard-code a number — just know that tool definitions are never free, and that a 30-tool payload resent on every loop iteration is a real line item.

A worked example

Take one tool, get_order_status, with two arguments: an order id string and an optional include_tracking boolean. Here is the same tool in both dialects, kept deliberately minimal.

Anthropic Messages — flat, and note input_schema:

tools = [{
  "name": "get_order_status",
  "description": "Look up the current status of a customer order by id. If the user has not given an order id, ask for it instead of guessing.",
  "input_schema": {
    "type": "object",
    "properties": {
      "order_id": {"type": "string"},
      "include_tracking": {"type": "boolean"}
    },
    "required": ["order_id"]
  }
}]

OpenAI Chat Completions — nested, and note parameters:

tools = [{
  "type": "function",
  "function": {
    "name": "get_order_status",
    "description": "Look up the current status of a customer order by id. If the user has not given an order id, ask for it instead of guessing.",
    "parameters": {
      "type": "object",
      "properties": {
        "order_id": {"type": "string"},
        "include_tracking": {"type": "boolean"}
      },
      "required": ["order_id"],
      "additionalProperties": false
    },
    "strict": true
  }
}]

Now the handling divergence. On OpenAI you read choice.message.tool_calls[0], take function.name, and run function.arguments through a JSON parser inside a try/except, because it is a string and a malformed string is a real outcome. On Anthropic you find the content block whose type is tool_use and read block.input directly as a dict. Then you send the result back: OpenAI as a message with role: "tool", tool_call_id set to the call id, and the payload in content; Anthropic as a user message containing a tool_result block with tool_use_id and the payload.

Two rules that belong in the wrapper, not in each handler. First, echo the full prior assistant turn back on the follow-up request — dropping the tool-call turn and sending only the result is the classic cause of the model re-requesting the same tool forever. Second, validate the parsed arguments against your own schema before you execute, even under strict mode, because strict mode is a property of the model call and not a property of the untrusted string that reached your database layer.

Failure modes that cost money

Hallucinated arguments. Anthropic documents this behaviour outright: when a required parameter is missing from the conversation, less capable models — Haiku-class, in Anthropic's own framing — may infer a value rather than stopping, while stronger models are more likely to ask the user for it. The mitigation is cheap and boring: state the ask-do-not-guess rule inside the tool description, keep required fields genuinely required, and reject on validation rather than executing a plausible-looking call.

Wrong tool selected. Overlapping descriptions are the cause more often than model weakness. Two tools called search_docs and search_kb with near-identical prose will be chosen at roughly coin-flip rates. Write descriptions that say when not to use a tool, and prune the tool list per turn — a 40-tool payload is both a cost and an accuracy problem.

Loops. The model calls a tool, dislikes the result, calls it again with a nudge, forever. Always cap iterations, and make the cap visible: on hitting it, return the partial state to the user rather than silently truncating. Every iteration resends the whole growing transcript, so an unbounded agent loop is the fastest way to turn a one-cent task into a one-dollar one.

Bad tool output. A 200 KB JSON blob returned as a tool result will blow context and degrade the answer. Summarise, paginate, or return ids and let the model ask for detail. Cost-per-task, not cost-per-token, is the number to optimise: a model that solves it in two tool calls at a higher per-token price frequently beats a cheap model that needs six and still guesses a field. That reframe is also the core of choosing a model for agent work.

Where MCP fits

Function calling defines how one model asks for one tool. It says nothing about where tool definitions come from, which is why every team ends up hand-wiring the same Slack, Postgres, and GitHub integrations again. MCP — the Model Context Protocol — is the standard for that layer. Anthropic introduced and open-sourced it in November 2024, with the first spec revision dated 2024-11-05, initial Python and TypeScript SDKs, and pre-built servers for Google Drive, Slack, GitHub, Git, Postgres, and Puppeteer; it has since been adopted well beyond Anthropic, including by OpenAI and Google DeepMind. Its own docs describe it as a USB-C port for AI applications: one protocol instead of N bespoke integrations.

Architecturally it is client-server over JSON-RPC 2.0. A server advertises its tools and their schemas through a discovery method, and the client invokes one through a call method — tools/list and tools/call respectively, with each listed tool carrying a name, a description, and an inputSchema. Check the revision you are targeting, since the spec is versioned and still moving. The key point for this article: MCP does not replace function calling, it feeds it. The MCP client fetches tool definitions at runtime and translates them into whatever shape the model API wants — parameters for OpenAI, input_schema for Anthropic — using exactly the mapping in the table above. If you are weighing the standard against direct integration, see MCP vs a plain API, and how to build an MCP server for the implementation side.

A/B tool quality across models

Tool-calling ability is not uniform, and it is not well predicted by general benchmark rank. The Berkeley Function Calling Leaderboard, an independent academic benchmark from the Gorilla project at UC Berkeley, evaluates it directly using a deterministic abstract-syntax-tree comparison that scales to thousands of candidate functions and covers both serial and parallel calls. Its versions track how the problem itself matured: v1 introduced the AST evaluation, v2 added enterprise and open-source contributed functions, v3 added multi-turn and multi-step, v4 moved toward holistic agentic evaluation including web search, memory, and format sensitivity. Leaderboard positions change constantly, so treat any specific percentage as a snapshot to re-verify on the official leaderboard rather than a stable fact.

Public leaderboards also cannot tell you how a model handles your schema, which is the only question that matters. The gateway version of this test is straightforward, and it is the thing no vendor doc frames because each doc covers only its own model: send the identical tool array and the identical prompt set through one OpenAI-compatible key to several models, then score three things.

Run twenty to fifty prompts per model and the ranking is usually unambiguous within an hour. Because DataLLM Lab is OpenAI-compatible, the harness is one client and a swapped model string — you can put Claude Opus 4.8, GPT-5.5, an open-weight option like GLM 5.2, and a cheap fast model such as DeepSeek V4 Flash through the same rubric, then check the per-model pricing to convert score into cost per completed task. That last conversion is the decision rule: pick the cheapest model whose argument-validity rate clears your validation-failure budget, not the highest-scoring one.

FAQ