LLM Tool Use and Function Calling#

LLM tool use function calling lets a model declare it wants to invoke a named function with structured arguments, instead of guessing the answer in prose. The runtime executes the function, feeds the result back, and the model continues.

flowchart LR
  M[LLM] -->|tool_call name + args JSON| R[Runtime]
  R -->|invoke| F[Function or API]
  F -->|result| R
  R -->|tool_result| M
  M -->|final text| U([User])

    classDef client fill:#dbeafe,stroke:#1e40af,stroke-width:1px,color:#0f172a;
    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    classDef external fill:#fce7f3,stroke:#9d174d,stroke-width:1px,color:#0f172a;
    class U client;
    class M service;
    class R compute;
    class F external;

The contract is a tool schema: name, description, and JSON Schema for the arguments. The model never executes anything itself; it only emits a structured call object. The runtime validates, dispatches, and returns the result. This separation keeps the LLM auditable and side-effect-free.

Modern models support parallel tool calls in one assistant turn (e.g. "look up user 42 and fetch their last order at the same time"), which collapses what used to be N round trips into one. The runtime executes the calls concurrently and returns all results in the next message.

The hard parts are not in the schema. They are in tool selection (picking the right tool from 20), argument quality (well-formed JSON that matches the schema), and recovery (turning a 500 into a useful next step rather than a panic loop). Tight tool descriptions and structured error returns matter more than the prompt.

Problem statement

Your agent needs to query a database, hit a third-party API, and run arithmetic. Plain prose responses cannot do any of this safely. Design a tool-calling layer that lets the LLM declare structured intent, lets your runtime execute that intent, and surfaces results back into the conversation in a way the next inference step can reason about.

LLM tool use function calling is the bridge between a language model's reasoning and the outside world. The model produces a JSON object naming a tool and its arguments; your runtime executes the tool and returns the result as the next user-role turn. The model itself remains a pure function from messages to messages.

Anatomy of a tool definition#

A tool has three things: a unique name, a natural-language description, and a JSON Schema for arguments. The description is what the model reads to decide whether to call the tool; treat it like API docs that a junior engineer will use without context.

get_weather_tool = {
    "name": "get_weather",
    "description": (
        "Get current weather for a city. Returns temperature in Celsius "
        "and a one-word condition (sunny, cloudy, rain, snow). "
        "Use when the user asks about weather, do not use for forecasts "
        "more than 1 hour out."
    ),
    "input_schema": {
        "type": "object",
        "properties": {
            "city": {
                "type": "string",
                "description": "City name, e.g. 'Tokyo' or 'San Francisco, CA'",
            },
            "units": {
                "type": "string",
                "enum": ["celsius", "fahrenheit"],
                "default": "celsius",
            },
        },
        "required": ["city"],
    },
}

JSON Schema features that move the needle: enum (restrict to a closed set), pattern (regex on strings), minimum/maximum, and required. Models respect these constraints reasonably well; validate on the runtime side regardless.

The request-response shape#

sequenceDiagram
  autonumber
  participant U as User
  participant R as Runtime
  participant L as LLM
  participant T as Tool
  U->>R: "What is the weather in Tokyo?"
  R->>L: messages + tools
  L-->>R: tool_use(name=get_weather, input={city:"Tokyo"})
  R->>T: GET /weather?city=Tokyo
  T-->>R: {temp: 21, cond: "cloudy"}
  R->>L: tool_result(content="21C cloudy")
  L-->>R: "It is 21C and cloudy in Tokyo."
  R-->>U: final text

The runtime appends both the assistant tool-call turn and the user tool-result turn before re-invoking the model. The model sees its previous call and the resulting data, then chooses whether to call again or answer.

Parallel tool calls#

When two tools are independent, modern models emit them in a single assistant turn:

# assistant content (Anthropic shape)
[
    {"type": "tool_use", "id": "t1", "name": "get_weather", "input": {"city": "Tokyo"}},
    {"type": "tool_use", "id": "t2", "name": "get_weather", "input": {"city": "Paris"}},
]

The runtime executes both concurrently and returns:

# next user content
[
    {"type": "tool_result", "tool_use_id": "t1", "content": "21C cloudy"},
    {"type": "tool_result", "tool_use_id": "t2", "content": "17C rain"},
]

flowchart LR
  M[LLM turn] --> P{Parallel?}
  P -- yes --> C1[Tool A]
  P -- yes --> C2[Tool B]
  P -- yes --> C3[Tool C]
  C1 --> J[Join results]
  C2 --> J
  C3 --> J
  J --> N[Next LLM turn]
  P -- no --> S[Single tool] --> N

    classDef service fill:#fef3c7,stroke:#92400e,stroke-width:1px,color:#0f172a;
    classDef compute fill:#d1fae5,stroke:#065f46,stroke-width:1px,color:#0f172a;
    class M,N service;
    class C1,C2,C3,S,J compute;

Parallel calls cut latency drastically when each tool is slow. Watch out: ordering of results is by tool_use_id, not array order, and the model expects all calls to come back before the next turn.

Provider-specific shapes#

Aspect	Anthropic	OpenAI	Gemini
Tool list key	`tools=[{name, description, input_schema}]`	`tools=[{type:"function", function:{name, description, parameters}}]`	`tools=[{function_declarations:[...]}]`
Call in response	`content` block `{type:"tool_use", id, name, input}`	`tool_calls[]` with `id`, `function.name`, `function.arguments` (JSON string)	`parts[]` with `functionCall:{name, args}`
Result back	`role:"user"` with `tool_result` block + `tool_use_id`	`role:"tool"` message with `tool_call_id` + content	`role:"function"` part with `functionResponse`
Parallel	Default-on	`parallel_tool_calls: true` flag	Supported, less common
Force a tool	`tool_choice: {type:"tool", name:"..."}`	`tool_choice: {type:"function", function:{name}}`	`function_calling_config:{mode:"ANY"}`
Streaming partial args	Delta events with `input_json_delta`	Delta with `arguments` chunks	Streaming chunks

The biggest footgun is OpenAI's arguments being a JSON string that you must json.loads(), while Anthropic returns a parsed dict. Wrap both behind one runtime adapter.

Error recovery patterns#

Tools fail. Return errors as tool results, not exceptions:

def safe_invoke(name, args):
    try:
        return {"ok": True, "data": dispatch(name, args)}
    except ValidationError as e:
        return {"ok": False, "error_type": "bad_args", "detail": str(e)}
    except HTTPError as e:
        return {"ok": False, "error_type": "upstream", "status": e.status, "detail": str(e)}
    except TimeoutError:
        return {"ok": False, "error_type": "timeout"}

The model can read error_type and decide whether to retry, change arguments, or fall back. Hiding errors behind generic strings ("something went wrong") destroys the model's ability to recover.

Tool selection failures#

When the LLM picks the wrong tool, the cause is almost always one of:

Overlapping descriptions: two tools that sound similar. Fix by adding "use this instead of X when..." in each description.
Too many tools: above 15 to 20 tools, accuracy drops. Group into sub-agents.
No negative examples: add "Do NOT use this for..." in the description.
Sparse arguments: a tool with no required args is called too eagerly. Add a required field that forces commitment.

A defensive runtime#

import json, jsonschema

class ToolRuntime:
    def __init__(self, tools):
        self.tools = {t["name"]: t for t in tools}

    def invoke(self, name: str, args: dict) -> dict:
        spec = self.tools.get(name)
        if not spec:
            return {"ok": False, "error_type": "unknown_tool", "name": name}
        try:
            jsonschema.validate(args, spec["input_schema"])
        except jsonschema.ValidationError as e:
            return {"ok": False, "error_type": "bad_args", "detail": e.message}
        try:
            result = self._dispatch(name, args)
            return {"ok": True, "data": result}
        except Exception as e:
            return {"ok": False, "error_type": "runtime", "detail": repr(e)}

Schema-validating on the runtime catches malformed JSON before it hits real systems. The "unknown_tool" branch handles model hallucinations of nonexistent tool names, which still happens at a low rate.

Structured output vs tool calling#

Tool calling and structured output (JSON mode, response_format with schema) are siblings. Use tool calling when the model needs to choose among actions; use structured output when there is exactly one shape the response must match. Tool calling is more expressive, structured output is cheaper and easier to validate.

Quick reference#

Mental model#

LLM emits a structured tool call object. Runtime executes. Result feeds back as next turn.

Tool definition#

Unique name
Natural-language description (the model's only docs)
JSON Schema for input (use enum, pattern, required)

Selection accuracy levers#

Tight, non-overlapping descriptions
"Use INSTEAD OF X when..." phrasing
Negative examples ("Do NOT use for...")
Keep tool count under 15 to 20
Required arguments to force commitment

Provider shapes#

	Anthropic	OpenAI	Gemini
Args	`dict`	JSON `string`, parse	`dict`
Call block	`tool_use`	`tool_calls[]`	`functionCall`
Result role	`user` + `tool_result`	`tool`	`function`
Parallel	default	flag	flag

Parallel calls#

Emit multiple tool_use in one assistant turn
Execute concurrently
Return all tool_result before next inference
Result match by tool_use_id

Error handling#

Never raise to the model
Return structured {ok:false, error_type, detail}
Error types: bad_args, upstream, timeout, unknown_tool, runtime
Schema-validate args before dispatch

Common bugs#

Hallucinated tool name (validate registry)
Malformed JSON (validate schema)
Eager dispatch of side-effect tools (require confirmation field)
Result too large (truncate or summarize before returning)

Tool calling vs structured output#

Tool calling: choose among actions
Structured output: exactly one response shape

Refs#

Anthropic, tool use docs
OpenAI, function calling guide
Gemini, function calling guide

FAQ#

What is LLM function calling?#

Function calling, also called tool use, lets the model emit a structured request to invoke a named function with typed arguments. The runtime executes the function and feeds the result back as another turn.

What is the difference between OpenAI function calling and Anthropic tool use?#

Both expose the same shape: a tool schema, a tool_use response, and a tool_result follow-up. Anthropic encourages multiple parallel tool calls more aggressively, and the response field names differ slightly.

How does parallel tool calling work?#

Modern models can emit multiple tool calls in one assistant turn when the calls are independent. The runtime fans out the executions, collects all results, and feeds them back together in a single follow-up turn.

How do you handle tool errors with LLMs?#

Return the error as a structured tool_result string, not as an exception. The model can then read the error and decide to retry, adjust arguments, or switch tools, which usually recovers gracefully.

How many tools should I expose to an LLM?#

Keep it small. Beyond 15 to 20 tools, models start picking the wrong one. Group related actions, use hierarchical tools, or route by intent before exposing a smaller relevant subset.

Model Context Protocol (MCP): standard transport for tool registries
Agent Loop and ReAct Pattern: the loop that orchestrates repeated tool calls
Vector Databases: retrieval is the most common tool an agent calls