Function Calling & Tool Use (2023) › Limitations

Despite its practical impact, function calling had several limitations. It required the model to correctly reason about when and how to use functions, and this reasoning could still fail in edge cases. The model might choose the wrong function, provide incorrect parameters, or fail to recognize situations where a function call was needed. These errors could lead to application failures or incorrect actions, requiring developers to build extensive error handling and validation logic.

The reliability of function calling depended on the quality of function descriptions provided to the model. If function schemas were ambiguous, incomplete, or poorly described, the model might misunderstand how to use them. Developers had to carefully craft function descriptions, parameter names, and documentation to ensure the model could use them correctly. This created an additional burden on developers and required expertise in prompt engineering and schema design.

Function calling also introduced new security and safety concerns. Unlike text generation, function calls could have real-world effects: they could modify data, make purchases, send emails, or perform other actions with consequences. This created risks of malicious use, accidental actions, or unauthorized access. Applications using function calling had to implement careful authorization and validation to ensure that function calls were appropriate and safe. The model itself could not always distinguish between safe and unsafe actions, requiring application-level safeguards.

The structured nature of function calling also limited flexibility. While structured outputs were valuable for reliability, they constrained the model's ability to express qualifications or context that did not fit neatly into predefined schemas. Applications requiring highly flexible or creative outputs might find function calling too rigid, preferring the free-form text generation that language models could provide without function constraints.

Another limitation was the overhead involved in function calling workflows. Each function call required multiple API interactions: sending the conversation with function schemas, receiving the function call request, executing the function, and sending results back to the model. This created latency and cost that might not be necessary for simple tasks that could be handled with direct text generation. Applications had to balance the benefits of function calling against these overheads, sometimes choosing simpler approaches for straightforward tasks.

The model's ability to reason about function calls was also limited by its training data and reasoning capabilities. If a function performed a task that was not well-represented in the model's training data, or if the task required specialized domain knowledge, the model might struggle to use the function appropriately. This limited function calling's effectiveness for highly specialized or novel applications that fell outside common use patterns.

Function calling also did not solve the fundamental limitation of language models regarding real-time information. While functions could provide current data, the model itself still worked with its training knowledge and might not understand recent events, changes in APIs, or domain-specific contexts that emerged after training. The model could call functions to get information, but it might not know what functions to call or what questions to ask if it lacked relevant context.