Function Calling & Tool Use (2023) › The Solution

Function calling provided a solution to these problems by introducing a standardized way for language models to interact with external functions and to produce structured outputs. The solution had two key components: a structured way to describe functions to the model, and a reliable mechanism for the model to request function calls that matched those descriptions exactly.

Function calling worked by allowing developers to provide function schemas to the language model using a structured format, typically JSON Schema. These schemas described the function name, purpose, and parameters with their types and constraints. The model could then reason about whether to call a function based on the conversation context, and if so, what parameters to use. The model's response would include a structured request to call a specific function with specific parameters, formatted according to the provided schema.

The technical implementation involved extending the language model's training and inference process to recognize function descriptions and to generate function call requests as a special type of output. When a model was presented with function schemas in the conversation, it learned to recognize situations where calling those functions would be helpful. Instead of just generating text, the model could generate a structured function call that the application could then execute programmatically.

The function calling mechanism used a structured output format that was separate from normal text generation. When the model decided to call a function, it would output a special token or format indicating a function call, followed by the function name and a JSON object containing the parameters. This structured format could be reliably parsed by the application, eliminating the need for brittle text parsing. The format was deterministic and schema-validated. This ensured that function calls matched the described interface exactly.

The system also included a mechanism for handling function responses. After the application executed a function call and obtained results, it could provide those results back to the model as tool responses. The model could then incorporate this information into its reasoning and generate a natural language response to the user that referenced the function results. This created a cycle: the model could reason about what information or actions it needed, request function calls to obtain them, process the results, and respond using that information.

OpenAI's implementation introduced several important design decisions that made function calling practical and reliable. The model was trained to recognize when function calling was appropriate based on context, rather than requiring explicit instructions in every prompt. It could handle multiple functions simultaneously, selecting the right one based on the user's request. The output format was strictly structured, using JSON that matched the provided schema, making parsing and validation straightforward for developers.

The system also supported a mode where the model could be instructed to always call a function for extracting structured information, even when a user's request was conversational. This enabled use cases like extracting entities from text, classifying content, or parsing user intents into structured formats. The model could act as a sophisticated parser, using its language understanding to extract structured data from natural language input with high reliability.