What Is Function Calling (Tool Use)? How LLMs Invoke External Tools

LLMs (large language models) like ChatGPT and Claude are good at producing text, but on their own they cannot perform actual actions such as “look up today’s weather” or “place an order in an internal system.” The mechanism that breaks through this wall and lets an LLM call external tools and APIs is function calling. Anthropic and others call the same mechanism tool use.

Function calling is a quiet but critically important technology that underpins the much-discussed AI agents and MCP. This article explains what function calling is, how an LLM invokes external tools, and its mechanics and use cases, based on Anthropic’s official documentation and other sources, for a non-specialist audience. Reading it alongside the basics of LLMs will deepen your understanding.

Why It’s Needed: An LLM Alone Has No Power to Act

An LLM is an AI specialized in predicting the continuation of given text to produce natural writing. It holds a vast amount of knowledge, but that knowledge stops at the point it was trained, and it has the limitation that it “cannot retrieve real-time information, perform calculations, or connect to external systems”³. For example, if you ask “What’s the weather in Tokyo today?”, the LLM itself cannot access weather data.

This is what function calling addresses. NTTPC’s explainer describes it as “a mechanism in which the LLM judges, from the user’s input, whether a specific tool (function) needs to be executed, and passes the necessary arguments to that function”². By letting the LLM use “external tools” such as a weather API, search, or an internal inventory system, tool use means “the LLM becomes able to call external tools, greatly expanding the range of practical applications”³. An AI that only writes text transforms into a system that can act—looking things up, performing calculations, and advancing procedures.

How It Works: The LLM Decides “Which Tool to Use,” and the App Executes

The most important point about function calling is that the LLM is not actually running the program. The roles are split into two.

According to NTTPC’s explainer, the rough flow is the following four steps².

The user gives an instruction (e.g., “Tell me three tourist spots in New York”)
The LLM selects the necessary function (e.g., a function to get tourist spots) and specifies the arguments
The function is executed and a result is obtained from the API
The LLM converts the result into natural language and replies

Of these, what the LLM handles is only step 2: deciding “which function to call, with what arguments.” Another technical explainer likewise frames it as the LLM performing “tool selection” and “parameter extraction,” while “the actual tool execution is handled on the application side”³.

The LLM returns this decision not as human-facing prose but in a fixed format called structured JSON. For example, it explicitly states the function name and arguments in a form like {"tool": "tool_name", "arguments": {"parameter_name": "value"}}, and the program that receives it actually executes the function—a division of labor³. The LLM acts as a “command center” issuing instructions on what to do, while the application serves as the “hands and feet” that carry out execution. This division of roles is the heart of function calling.

Client Tools and Server Tools

Anthropic’s official documentation organizes tools into two types based on where they execute.

One is client tools, which run on the app side. When Claude judges that it should use a tool, it returns stop_reason: "tool_use" (a signal that it stopped to use a tool) along with a tool_use block describing what to call. Your app’s code executes that operation and returns the result as a tool_result, after which Claude continues its answer based on it¹. This applies when you want to let it use your own inventory system or proprietary API.

The other is server tools, such as web search and code execution, which run on Anthropic’s infrastructure and return results directly¹. There is no need to write execution code on the app side.

The decision of whether to call a tool can also be controlled. By default, tool_choice is set to auto, and Claude decides on each turn whether to “call a tool or answer directly.” It calls a tool when the request matches the tool’s capability and the answer is not already in the information at hand, and it answers directly for stable knowledge, creative tasks, and conversational exchanges¹. Note that adding strict: true to a tool definition guarantees that Claude’s tool calls always conform exactly to the specified schema (format)¹.

How It Differs from RAG

RAG (retrieval-augmented generation) is another well-known technology for letting an LLM use external information. The two are similar but distinct, with different purposes.

In NTTPC’s framing, the purpose of function calling is to “execute a process,” while the purpose of RAG is to “improve output accuracy”². Whereas RAG is a mechanism that finds relevant information from internal documents and FAQs to raise the accuracy of an LLM’s answers, function calling is a mechanism that actually executes something, such as placing an order or making a reservation.

The two are not mutually exclusive, and combining them is effective. NTTPC gives the example of “combined operation,” such as searching inventory information with RAG and then automatically ordering the shortfall with function calling based on that result². By linking looking things up (RAG) with taking action (function calling), you can build more practical AI applications.

As the Foundation of AI Agents and MCP

The importance of function calling lies less in its own convenience and more in the fact that it has become the foundation supporting recent trends in AI adoption.

AI agents, in which an LLM advances tasks autonomously, repeat the action of “looking at the situation, deciding which tool to use next, executing it, looking at the result, and thinking about what to do next.” What makes each of these tool calls possible is function calling. Anthropic itself positions making tools available as one of the highest-leverage things you can give an agent¹.

Likewise, MCP (Model Context Protocol), which standardizes connections to tools and data sources, ultimately rests on the idea of function calling for the part where the LLM invokes a tool. When following topics like AI agents and MCP, knowing that this division of labor—“the LLM judges which tool to use and how, and the app handles execution”—sits at the root makes it easier to see what each technology is solving.