What is tool calling? Function calling in LLMs, explained
Tool calling lets an LLM ask your code to run a function with JSON arguments. The request and result shapes, error handling, and a runnable Python demo.

Tool calling (also called function calling) is how a language model asks your application to run a function: you describe the tools with a name, a description and a JSON schema, the model replies with a structured call naming one tool and its arguments, your code runs it, and you send the result back. The model never executes anything itself. It only writes a request in a format your code can parse, which is what makes an AI agent possible.
Where did tool calling come from?
Two lines of work met in 2023. On the research side, Toolformer (Schick et al., February 2023) trained a model to decide which APIs to call, when, and with what arguments, using a calculator, a Q&A system, search engines, a translator and a calendar. On the product side, OpenAI shipped function calling in the Chat Completions API on June 13, 2023, with the gpt-4-0613 and gpt-3.5-turbo-0613 snapshots fine-tuned to detect when a function should be called and to answer with JSON that matches its signature.
Every major model API now has the same feature under slightly different names. Anthropic’s docs put it plainly: “Tool use (also called function calling) lets Claude call functions that you define or that Anthropic provides.”
How does tool calling work?
Using the Claude Messages API as the reference, one round trip has four steps:
- You send tools with the request. Each has a
name, adescriptionand aninput_schemain JSON Schema. - The model replies with a call. The response has
stop_reason: "tool_use"and one or moretool_useblocks, each with anid, the toolnameand aninputobject. - Your code runs the function. This is ordinary code in your process: an HTTP call, a SQL query, a file read.
- You send a
tool_resultback in the next user message, withtool_use_idmatching the call, the output ascontent, andis_error: trueif it failed. The model then answers or calls another tool.
Run steps 2 to 4 in a loop and you have an agent. Stop after one round and you have a single tool-assisted answer.
Anthropic separates client tools, which run in your application, from server tools such as web search, web fetch and code execution, which run on Anthropic’s infrastructure and come back in the same response. For your own functions, the loop above is always your job.
What does a tool call look like in code?
The model side needs an API key, so this script covers the other half, the part you write. It turns a Python function into a tool definition, takes tool_use blocks shaped like a model’s reply (two parallel calls, one of them bad), validates and runs them, and builds the tool_result message that goes back.
# tool_calling.py: the application side of tool calling, with no model and no network.
# Python 3 stdlib only. 1) turn a Python function into a JSON Schema tool definition,
# 2) take tool_use blocks shaped like a model's reply, 3) validate, run, return tool_result.
import inspect, json
def get_fx_rate(base: str, quote: str) -> float:
"""Return today's exchange rate from base to quote currency, e.g. USD to INR.
Use it whenever the user asks to convert an amount between currencies."""
rates = {("USD", "INR"): 88.2, ("EUR", "INR"): 102.5} # stand-in for a real API
if (base, quote) not in rates:
raise ValueError(f"no rate for {base}->{quote}; supported: USD->INR, EUR->INR")
return rates[(base, quote)]
JSON_TYPES = {str: "string", float: "number", int: "integer", bool: "boolean"}
def to_tool(fn):
params = inspect.signature(fn).parameters
return {"name": fn.__name__,
"description": inspect.getdoc(fn),
"input_schema": {"type": "object",
"properties": {p: {"type": JSON_TYPES[v.annotation]} for p, v in params.items()},
"required": [p for p, v in params.items() if v.default is inspect._empty]}}
REGISTRY = {"get_fx_rate": get_fx_rate}
tool = to_tool(get_fx_rate)
print("name:", tool["name"])
print("input_schema:", json.dumps(tool["input_schema"]), "\n")
def run_tool_use(block):
schema = to_tool(REGISTRY[block["name"]])["input_schema"]
missing = [k for k in schema["required"] if k not in block["input"]]
try:
if missing:
raise ValueError(f"missing required argument(s): {missing}")
content, is_error = str(REGISTRY[block["name"]](**block["input"])), False
except Exception as e: # errors go back to the model, not up the stack
content, is_error = f"{type(e).__name__}: {e}", True
return {"type": "tool_result", "tool_use_id": block["id"], "content": content, "is_error": is_error}
# What a model's reply might contain: two parallel calls, the second one wrong.
assistant_content = [
{"type": "tool_use", "id": "toolu_01", "name": "get_fx_rate", "input": {"base": "USD", "quote": "INR"}},
{"type": "tool_use", "id": "toolu_02", "name": "get_fx_rate", "input": {"base": "GBP", "quote": "INR"}},
]
next_user_message = {"role": "user", "content": [run_tool_use(b) for b in assistant_content]}
for r in next_user_message["content"]:
print(json.dumps(r))
Output (Python 3.14.5):
name: get_fx_rate
input_schema: {"type": "object", "properties": {"base": {"type": "string"}, "quote": {"type": "string"}}, "required": ["base", "quote"]}
{"type": "tool_result", "tool_use_id": "toolu_01", "content": "88.2", "is_error": false}
{"type": "tool_result", "tool_use_id": "toolu_02", "content": "ValueError: no rate for GBP->INR; supported: USD->INR, EUR->INR", "is_error": true}
Three details carry most of the weight. The docstring becomes the description, which is the text the model reads to decide whether to call the tool. The error is caught and returned as a result instead of crashing the loop, and it says what is supported, so the model can recover. And both results go back in one user message, matched by tool_use_id. The rate is a hard-coded stand-in, not a real quote.
What makes a good tool definition?
Anthropic’s define tools page is blunt: extremely detailed descriptions are “by far the most important factor in tool performance.” It asks for what the tool does, when to use it and when not to, what each parameter means and what the tool does not return, and to aim for at least three to four sentences per tool. Its other advice, condensed:
- Consolidate. One tool with an
actionparameter beatscreate_pr,review_prandmerge_pras three tools. - Namespace. Prefix names by service, such as
github_list_prsorslack_send_message, once you have many. - Return high-signal results. Stable IDs and only the fields the model needs; bloated responses waste context.
- Write instructive errors. “Rate limit exceeded. Retry after 60 seconds.” beats “failed”.
On the handle tool calls page, Anthropic says that for an invalid call Claude “will retry 2-3 times with corrections before apologizing to the user”, which is why the error text is worth writing carefully.
How do you control when the model calls a tool?
Most APIs have a tool choice setting. Claude’s tool_choice has four values: auto (the model decides, the default when tools are present), any (it must use one of the tools), tool (it must use a named tool) and none. The docs note that the newest models, including Claude Opus 5.5, reject forced any and tool with a 400 error and point you to auto plus strict tool use instead, so check the model before relying on forcing.
Strict mode guarantees the arguments match your schema. OpenAI added strict: true on function definitions with Structured Outputs in August 2024; Anthropic’s tool definitions accept strict: true as well. Without it, the model usually follows the schema; with it, decoding is constrained so it always does.
Tools cost tokens. Definitions are input tokens on every request, and the API adds its own tool-use system prompt. As of October 2026, Anthropic’s table lists 496 extra tokens for Claude Opus 4.5 and 286 for Opus 5.5 with tool_choice set to auto, before your own definitions. Forty tools with long descriptions add up fast, which is the problem code execution with MCP tries to work around.
Tool calling vs MCP: what’s the difference?
Tool calling is the model API feature: how one request describes functions and how the model asks for one. MCP is a protocol for where those functions come from: a host discovers tools on separate servers over JSON-RPC and exposes them to the model, which still picks them through tool calling. You can use tool calling without MCP by defining functions inline, and every MCP host uses tool calling underneath.
Where does tool calling go wrong?
- The model guesses missing arguments. Anthropic notes that when a required parameter isn’t in the prompt, Claude Opus is more likely to ask, while Sonnet “might also infer a reasonable value.” Validate anything that matters.
- Tool results are untrusted. Anthropic’s docs warn that tool results often carry content from outside your control, and that an attacker can embed instructions in it. Keep that content inside
tool_resultblocks, never in the system prompt. Background in what prompt injection is. - Message ordering. Results must come right after the assistant turn that called them, and
tool_resultblocks must come first in the user message. Get it wrong and the API returns a 400.
What I would do
Write the tool layer as if a new colleague will read only the descriptions, because that’s close to what happens. Keep the set small, return errors as results with a next step in the text, turn on strict mode for anything you parse downstream, and log every call with its arguments and result. For anything that writes, sends or pays, check the arguments in code before running it, whatever the model says.
FAQ
Is function calling the same as tool calling?
Yes. OpenAI called it function calling in 2023; Anthropic and others say tool use or tool calling. The mechanism is the same: a schema in, a structured call out.
Does the LLM execute the function?
No. The model only returns the function name and arguments. Your code, or the provider’s servers for hosted tools, runs it and sends back the result.
Can a model call several tools at once?
Yes. Claude and OpenAI models can return several calls in one turn (parallel tool use). Return all the results together in the next message, each matched by its call ID.
What is the difference between tool calling and structured output?
Tool calling produces a call meant to be executed, then continues the conversation. Structured output makes the final answer itself match a JSON schema. Both can use the same constrained decoding underneath.
Sources
- Anthropic, Tool use with Claude, Define tools and Handle tool calls, as of October 31, 2026
- OpenAI, Function calling and other API updates, June 13, 2023
- OpenAI, Introducing Structured Outputs in the API, August 6, 2024
- Schick et al., Toolformer: Language Models Can Teach Themselves to Use Tools, arXiv, February 9, 2023