Function Calling: Structured Tool Use for Large Language Models › Code Implementation: Building a Function Calling System
Language AI HandbookAll chapters Function Calling: Structured Tool Use for Large Language Models › Code Implementation: Building a Function Calling SystemOne section · Sections in this chapter6 / 12 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Let's implement a complete function calling pipeline using Python. We'll create a mock LLM interface that simulates generation, then build the execution framework around it. This implementation demonstrates the architectural patterns used in production systems, abstracted for clarity.
The architecture we build here separates three distinct concerns. The tool registry manages the available capabilities: it stores schemas (the contract with the LLM) and implementations (the actual executable code). The mock LLM encapsulates the decision logic: which tool to call and with what arguments. The agent orchestrates the interaction: it maintains conversation state, sends queries to the LLM, executes tool calls, and feeds results back into the conversation. In a production system, the mock LLM would be replaced by real API calls, but the registry and agent patterns remain unchanged. This separation of concerns is what makes function calling systems maintainable as they grow: you can swap out the LLM backend without touching the tool implementations, or add new tools without modifying the orchestration logic.
from dataclasses import dataclass
from typing import Any, Dict, List
@dataclass
class FunctionCall:
name: str
arguments: Dict[str, Any]
call_id: str
@dataclass
class Message:
role: str
content: str = None
tool_calls: List[FunctionCall] = None
tool_call_id: str = None
name: str = NoneFirst, we define a registry for available tools. This registry maps function names to implementations and maintains their schemas. The registry pattern centralizes tool management. This provides a single source of truth for both the schemas (consumed by the LLM) and the implementations (consumed by the execution environment).
from typing import Callable
class ToolRegistry:
def __init__(self):
self.tools: Dict[str, Dict] = {}
self.implementations: Dict[str, Callable] = {}
def register(self, name: str, schema: Dict, implementation: Callable):
self.tools[name] = schema
self.implementations[name] = implementation
def get_schemas(self) -> List[Dict]:
return [
{"type": "function", "function": schema}
for schema in self.tools.values()
]
def execute(self, call: FunctionCall) -> Any:
if call.name not in self.implementations:
raise ValueError(f"Unknown function: {call.name}")
func = self.implementations[call.name]
try:
result = func(**call.arguments)
return {"status": "success", "result": result}
except Exception as e:
return {"status": "error", "error": str(e)}The execute method wraps the function call in a try-except block and returns a structured result with a status field. This pattern ensures that even when tools fail, the return value is parseable JSON that the model can reason about. A failed call that returns {"status": "error", "error": "Location not found"} gives the model actionable information; a failed call that raises an unhandled exception gives it nothing.
Now we implement the tools. We'll create a weather lookup and a calculator, simulating the weather API while implementing the calculator. This mixed approach is typical in development environments, where some tools connect to real services while others use mocks or local implementations.
def mock_weather_api(location: str, unit: str = "celsius") -> dict:
"""Simulated weather API for demonstration"""
weather_db = {
"boston, ma": {
"temp": 22,
"condition": "Partly cloudy",
"humidity": 65,
},
"san francisco, ca": {"temp": 18, "condition": "Foggy", "humidity": 80},
"london, uk": {"temp": 15, "condition": "Rainy", "humidity": 90},
}
key = location.lower().strip()
if key not in weather_db:
return {"error": f"No data available for {location}"}
data = weather_db[key]
if unit == "fahrenheit":
data = {**data, "temp": data["temp"] * 9 / 5 + 32}
return data
def calculator_tool(expression: str) -> float:
"""Safe calculator using AST-based expression evaluation.
All nodes are validated against an allowlist before any evaluation occurs."""
allowed_ops = {
ast.Add: operator.add,
ast.Sub: operator.sub,
ast.Mult: operator.mul,
ast.Div: operator.truediv,
ast.USub: operator.neg,
}
def eval_node(node):
if isinstance(node, ast.Constant):
if not isinstance(node.value, (int, float)):
raise ValueError("Only numeric constants allowed")
return node.value
elif isinstance(node, ast.BinOp):
op_type = type(node.op)
if op_type not in allowed_ops:
raise ValueError(f"Disallowed operator: {op_type.__name__}")
return allowed_ops[op_type](
eval_node(node.left), eval_node(node.right)
)
elif isinstance(node, ast.UnaryOp) and isinstance(node.op, ast.USub):
return -eval_node(node.operand)
raise ValueError(f"Disallowed node type: {type(node).__name__}")
tree = ast.parse(expression, mode="eval")
return eval_node(tree.body)We instantiate the registry and register our tools:
registry = ToolRegistry()
weather_schema = {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City and state/country",
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"default": "celsius",
},
},
"required": ["location"],
},
}
calc_schema = {
"name": "calculate",
"description": "Calculate mathematical expressions",
"parameters": {
"type": "object",
"properties": {
"expression": {
"type": "string",
"description": "Math expression to evaluate",
}
},
"required": ["expression"],
},
}
registry.register("get_weather", weather_schema, mock_weather_api)
registry.register("calculate", calc_schema, calculator_tool)For demonstration purposes, we'll simulate the LLM's function calling behavior with rule-based responses. In practice, this would be replaced with actual API calls to GPT-4, Claude, or open-source models with function calling capabilities. This mock implementation illustrates the expected interface: the LLM receives conversation history and returns either a text response or a structured tool call.
import re
class MockLLM:
"""Simulates an LLM with function calling capabilities"""
def __init__(self, registry: ToolRegistry):
self.registry = registry
self.call_count = 0
def generate(self, messages: List[Message]) -> Message:
"""Simulate generation based on message history"""
last_message = messages[-1]
content = last_message.content.lower() if last_message.content else ""
# Simple pattern matching to simulate tool use decisions
if "weather" in content:
self.call_count += 1
# Extract location with simple regex
match = re.search(r"weather\s+(?:in|for)?\s+(.+?)(?:\?|$)", content)
if match:
location = match.group(1).strip()
return Message(
role="assistant",
tool_calls=[
FunctionCall(
name="get_weather",
arguments={"location": location},
call_id=f"call_{self.call_count}",
)
],
)
elif any(
word in content
for word in ["calculate", "compute", "sum", "product", "how many"]
):
self.call_count += 1
# Look for math expressions
numbers = re.findall(r"\d+", content)
if len(numbers) >= 2 and "sum" in content:
return Message(
role="assistant",
tool_calls=[
FunctionCall(
name="calculate",
arguments={
"expression": f"{numbers[0]} + {numbers[1]}"
},
call_id=f"call_{self.call_count}",
)
],
)
# Default response if no tool needed
return Message(
role="assistant", content="I don't need any tools to answer this."
)Now we implement the execution loop that orchestrates the interaction between the LLM and tools. This agent class encapsulates the state management and control flow, maintaining the conversation history and managing the iterative cycle of generation and execution.
import json
class FunctionCallingAgent:
def __init__(
self, llm: MockLLM, registry: ToolRegistry, max_iterations: int = 5
):
self.llm = llm
self.registry = registry
self.max_iterations = max_iterations
self.conversation: List[Message] = []
def run(self, user_query: str) -> str:
"""Execute the full function calling loop"""
# Add system message with tool descriptions
system_msg = f"You have access to the following tools: {json.dumps(self.registry.get_schemas())}"
self.conversation = [Message(role="system", content=system_msg)]
self.conversation.append(Message(role="user", content=user_query))
for iteration in range(self.max_iterations):
# Generate response
response = self.llm.generate(self.conversation)
# Check if tool calls were made
if response.tool_calls:
self.conversation.append(response)
# Execute all tool calls (parallel execution)
for call in response.tool_calls:
result = self.registry.execute(call)
# Add observation to conversation
obs_msg = Message(
role="tool",
content=json.dumps(result),
tool_call_id=call.call_id,
name=call.name,
)
self.conversation.append(obs_msg)
# Continue loop for final generation
continue
else:
# No tool calls, return final answer
return response.content
return "Maximum iterations reached without final answer."agent = FunctionCallingAgent(MockLLM(registry), registry)
agent_result = agent.run("What's the weather in Boston, MA?")Final result: I don't need any tools to answer this.
The execution trace shows the complete cycle: the agent recognizes the weather query, extracts the location parameter, executes the mock API, and would typically generate a final response (in this simulation, the mock LLM returns a placeholder). This architecture separates concerns cleanly: the agent manages state and orchestration, the registry handles tool management, and the LLM handles decision-making. Such separation simplifies extension and testing as well as maintenance of production function calling systems.
The max_iterations guard deserves particular attention. Without it, a malfunctioning model that repeatedly generates tool calls without ever creating a final answer would loop forever, consuming API credits and blocking other requests. The iteration limit is a safety valve, not an expected boundary condition. Well-calibrated models almost never reach it. When they do, it is often a signal that the query is ambiguous, the tool outputs are confusing the model, or the model is stuck in a pattern matching loop. Logging iteration counts in production helps identify these pathological cases.