Tool Selection for LLM Agents: Routing Strategies and Implementation › Multi-Tool Scenarios
Language AI HandbookAll chapters Tool Selection for LLM Agents: Routing Strategies and Implementation › Multi-Tool ScenariosOne section · Sections in this chapter4 / 11 ↓
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
Real-world tasks rarely require just one tool. Complex workflows demand sequential tool use, parallel invocations, or conditional branching based on intermediate results. These multi-tool scenarios introduce combinatorial complexity: the model must select appropriate tools, determine their execution order, handle data dependencies, and manage failure modes across the chain.
The basic shift in multi-tool scenarios is from selection to planning. Single-tool routing is a one-shot classification problem: given a query, output a tool. Multi-tool routing is closer to program synthesis: given a query, output a sequence of tool calls that collectively accomplish the goal. The model must reason about what information each step produces and what the next step requires, maintaining a mental model of state as execution proceeds.
Sequential Tool Use
When tools have dependencies, they must execute in sequence. For example, to answer "What's the weather like at the location of my 3pm meeting?", the system must:
- Query the calendar to find the meeting location
- Query the weather service for that specific location
This creates a tool chain where the output of tool becomes input to tool . The LLM must recognize this dependency pattern during the initial selection phase and plan accordingly, understanding that the weather tool cannot be called until the calendar tool provides the location parameter.
This dependency structure is often invisible in natural language. The user does not say "first get the location, then get the weather"; they expect the system to infer this ordering from the semantic structure of the request. Models trained on sufficient examples of chained tool use develop a kind of procedural understanding: they learn that certain types of queries require information gathering before information computation, and that certain parameters cannot be filled until upstream tools have executed.
class ToolChain:
def __init__(self):
self.memory = {} # Store intermediate results
def execute_sequential(self, steps):
"""
Execute a sequence of tool calls where later steps
depend on earlier results.
"""
results = []
for step in steps:
tool_name = step["tool"]
# Allow referencing previous results via template
params = self._resolve_params(step["params"])
result = self._call_tool(tool_name, params)
self.memory[step["id"]] = result
results.append(result)
return results
def _resolve_params(self, params):
"""Replace template variables with actual values from memory."""
resolved = {}
for key, value in params.items():
if isinstance(value, str) and value.startswith("$"):
var_name = value[1:]
resolved[key] = self.memory.get(var_name, value)
else:
resolved[key] = value
return resolved
def _call_tool(self, tool, params):
# Mock implementation
return f"{tool}({params})"The _resolve_params method implements a simple template substitution: when a parameter value starts with "$", it is treated as a reference to a previous result stored in memory. This allows the LLM to express data dependencies declaratively in its output, rather than requiring it to embed actual values before they are known.
In more sophisticated implementations, the memory store becomes a scratchpad where intermediate results are indexed by step name. The LLM can then reference specific fields extracted from a previous tool output: "$calendar_lookup.location" might reference only the location field of the calendar result rather than the entire JSON object. This fine-grained referencing reduces the amount of context passed to each tool and makes the data flow explicit.
Parallel Tool Selection
Some tasks benefit from invoking multiple tools simultaneously. If a user asks "Compare the weather in New York and London," the system should recognize that two independent weather queries can execute in parallel, reducing total latency.
Modern LLMs support parallel function calling, generating multiple tool invocations in a single response. This requires the selection mechanism to identify independence: tools that do not share input dependencies and do not have side effects that interfere with each other. Determining independence requires analyzing data flow graphs and understanding which tools read versus write shared resources.
The independence check is subtler than it appears. Two weather tool calls for different cities are clearly independent. But two calls to a "search" tool might not be if they are drawing from a rate-limited API that throttles concurrent requests. And two calls to a "database_write" tool are never independent because they might contend for the same record. A reliable parallelism framework requires identifying data dependencies along with resource contention and side-effect semantics, information that must be encoded in tool metadata beyond the basic schema.
Parallel tool calling also affects error handling. When sequential tools fail, recovery is straightforward: you know exactly which step failed and what state was accumulated before the failure. When parallel tools fail, you must handle partial results: some tools may have succeeded while others failed, and the appropriate recovery strategy depends on which combination succeeded. Systems that support parallel tool calling need explicit failure policies: "all-or-nothing" (retry everything if any tool fails), "best-effort" (use whatever succeeded), or "conditional" (retry only failed steps with their dependencies).
Conditional and Branching Selection
Advanced scenarios require conditional logic: "If the meeting room is booked, find an alternative time." This introduces tool selection as planning, where the model must generate a program or plan tree rather than a flat sequence. The selection mechanism must evaluate runtime conditions and dynamically choose branches based on intermediate results.
The ReAct pattern (Reasoning + Acting), which we explore in the next chapter, addresses this by interleaving reasoning traces with tool invocations. Each reasoning step evaluates the current state and determines the next tool selection dynamically, rather than planning the entire sequence upfront. This reactive approach handles uncertainty and failure gracefully, letting the system to backtrack or pivot when tools return unexpected results.
The key insight behind ReAct is that planning and execution should be interleaved, not separated. A system that generates a complete plan before executing any tool is betting that all assumptions about tool outputs are correct. When they are wrong, the entire plan may need to be discarded. A system that reasons between each tool call can adapt: if the calendar tool returns "room unavailable," the next reasoning step can decide to call the "find_alternative_room" tool, rather than trying to anticipate this possibility in the original plan.
This adaptive behavior requires the LLM to maintain coherent state across multiple turns of tool invocation and reasoning. The model must keep track of what has been tried, what succeeded, what failed, and what remains to be done. This is a form of working memory management, and it represents one of the harder challenges in building reliable multi-tool agents. Models with larger context windows and stronger instruction-following capabilities tend to perform better at this kind of stateful planning.