Function Calling & Tool Use (2023) › The Problem

Before function calling, language models faced a fundamental limitation that prevented them from being truly useful in many practical applications. Models could generate text that described actions or provided information, but they could not perform those actions or access real-time information. This gap between language capability and practical utility created a frustrating divide between what language models could theoretically help with and what they could accomplish.

Consider a scenario where a user wanted a language model to check the weather, book a flight, or query a database. The model could generate text that looked like it was doing these things. It might say "I would check the weather for you" and then generate a plausible weather forecast, but this would be completely fabricated based on training data patterns, not actual current weather information. Similarly, the model might generate text that resembled a flight booking confirmation, but this would be meaningless text, not an actual booking. Users had to manually execute these actions themselves, using the model's text output as a guide but doing the actual work through separate interfaces.

The problem extended to structured data extraction as well. If a user needed to extract specific information from a conversation and use it programmatically, they faced the challenge of parsing natural language output that might be formatted inconsistently. The model might say "The user's email is john@example.com" in one response and "Contact: john@example.com" in another. Applications requiring structured data had to rely on brittle text parsing that could easily fail with slight variations in formatting. This made it difficult to build reliable integrations between language models and other software systems.

For developers building AI-powered applications, the inability to get structured outputs from language models created significant friction. They had to write complex parsing logic to extract information from text responses, deal with various formatting inconsistencies, and handle edge cases where the model's output didn't match expected patterns. This added development overhead and introduced points of failure in applications that relied on language model outputs. The lack of structured interfaces also made it difficult to chain multiple operations together reliably, limiting the complexity of tasks that could be automated.

Another critical limitation was the model's inability to access real-time or contextual information. A language model trained on data up to a certain date could not know about events, prices, or system states that occurred after its training cutoff. This made language models unsuitable for applications requiring current information, such as checking stock prices, retrieving user account details, or querying live databases. The model could only work with what was in its training data, creating a fundamental constraint for many practical applications.

The development of agent frameworks prior to 2023 had begun to address some of these issues through techniques like chain-of-thought reasoning and tool description in prompts. However, these approaches were unreliable. Models might misunderstand tool descriptions, fail to format outputs correctly, or hallucinate function calls that didn't match the actual API structure. The lack of a standardized way to describe functions and to ensure models produced valid, executable outputs prevented these approaches from becoming production-ready. Developers were forced to build extensive error handling and retry logic, making agent systems complex and fragile.