Tool Selection for LLM Agents: Routing Strategies and Implementation › Tool Descriptions as Interfaces

The relationship between an LLM and its available tools is mediated entirely through descriptions. Unlike software APIs where function signatures provide strict type contracts and compilation enforces correctness, LLMs interpret tools through natural language and structured schemas. This creates both extraordinary flexibility and inherent fragility: descriptions must be precise enough to guide correct selection, yet general enough to cover diverse use cases. The model cannot execute a tool to see what it does; it must infer functionality from text alone, much like a human reading a manual without ever touching the device.

This textual interface creates a basic design tension. If descriptions are too vague, the model cannot distinguish between similar tools. If they are too specific, they may fail to cover legitimate use cases that fall outside the narrow examples provided. Crafting effective descriptions requires understanding how language models interpret text: they rely on distributional semantics, where meaning emerges from patterns in training data rather than grounded experience.

The quality of tool descriptions has an outsized impact on routing accuracy. A study on GPT-4 function calling found that improving description clarity alone increased correct tool selection by 15-20 percentage points, without any changes to the underlying model. This makes description engineering one of the highest-impact activities when building tool-augmented systems. Developers who treat descriptions as afterthoughts, writing one-line summaries like "Weather tool" or "Get data," find that their systems make systematic routing errors that no amount of prompt tuning elsewhere can fix.

The most common approach to tool description follows the JSON Schema specification, which we encountered in our discussion of function calling. This standard provides a structured yet flexible way to define tool interfaces. Each tool is defined by three needed components:

• Name: A unique identifier that the model uses to reference the tool in its output. The name should be descriptive but concise, serving as a memorable handle for the capability.
• Description: A natural language explanation of what the tool does, when to use it, and when not to use it. This field is the primary semantic signal for selection.
• Parameters: A structured schema defining required and optional inputs, including types and constraints plus descriptions for each argument.

The description field is the primary signal for selection because it bridges the gap between user intent and tool capability. Research has shown that models are highly sensitive to phrasing variations, a phenomenon rooted in how attention mechanisms weight token relationships. A description like "Search the web for current information" elicits different behavior than "Retrieve up-to-date facts from the internet," even when referring to the same underlying capability. The former emphasizes the search action and recency, while the latter suggests fact retrieval, potentially triggering different patterns in the model's associative memory.

{
  "name": "weather_lookup",
  "description": "Retrieve current weather conditions and forecasts for a specific location. Use this when the user asks about temperature, precipitation, or weather conditions.",
  "parameters": {
    "type": "object",
    "properties": {
      "location": {
        "type": "string",
        "description": "City and country, e.g., 'Paris, France'"
      },
      "days": {
        "type": "integer",
        "description": "Number of days for forecast (1-7)",
        "default": 1
      }
    },
    "required": ["location"]
  }
}

The parameter descriptions are equally necessary to successful tool selection. When a model hesitates between two similar tools, subtle differences in parameter definitions often break ties. For instance, if one calculator tool accepts "expression" (mathematical notation) while another accepts "query" (natural language math), the parameter description guides the model toward the appropriate choice based on input format. This distinction helps the model route "2+2" to the expression-based calculator and "what is two plus two" to the natural language variant, even when both tools perform arithmetic.

Naming conventions matter more than developers typically expect. A tool named get_current_weather is more likely to be invoked for weather queries than one named atmospheric_conditions_retriever, even if they are functionally identical, because the former's name pattern is more commonly associated with weather-related contexts in training data. This means you should favor common, domain-appropriate naming over clever or technically precise identifiers. The model's internal associations are built on statistical patterns from human-written code and documentation, not from first principles.

Beyond basic schemas, effective tool descriptions often include few-shot examples showing proper usage. These examples anchor the model's understanding through concrete instances, showing what the tool does and the boundary conditions of when to invoke it. This approach uses the few-shot learning capabilities of transformer models, where performance improves dramatically with task-specific examples in the context.

Consider the distinction between these two description strategies for a calendar tool:

Minimal description:

Rich description with examples:

The rich description provides negative examples (when not to use the tool), which helps reduce false positives by defining decision boundaries. It also establishes relationships between tools, implicitly defining the routing policy through description content rather than explicit logic. This contextual steering is particularly important in crowded tool ecosystems where multiple tools might plausibly handle a request.

Negative examples deserve particular emphasis because they address one of the most common failure modes in production systems: false positives. Without guidance about when not to invoke a tool, models tend to over-trigger on tools whose names or descriptions partially match the query. A calendar tool might get invoked when the user simply asks "what day is it?" because both the query and the description contain temporal concepts. Explicit negative examples prevent this kind of shallow matching by forcing the model to distinguish between related but distinct capabilities.

The examples you choose should reflect the actual distribution of queries your system will receive, not idealized textbook cases. If your users frequently ask questions in informal language ("whats the weather tmrw?"), your description examples should include informal phrasings, not just grammatically pristine sentences. This alignment between training distribution and deployment distribution is a core principle of machine learning, and it applies equally to the "micro-training" that few-shot examples provide within the context window.

When the number of available tools grows large, including all descriptions in the context window becomes impractical or impossible. As we discussed in Part XLIV: Retrieval-Augmented Generation, we can treat tool selection as a retrieval problem, applying the same principles used for document retrieval to the domain of capability discovery.

In this paradigm, each tool description is embedded into a vector space using a sentence encoder. When a user query arrives, we embed the query and retrieve the $k$ most similar tool descriptions via vector similarity. Only these top-$k$ candidates are presented to the LLM for final selection. This approach turns an intractable $N$-way classification problem into a manageable $k$-way decision, where $k \ll N$.

This approach, sometimes called embedding-based routing, reduces context length and focuses the model's attention on relevant capabilities. However, it introduces a retrieval bottleneck: if the embedding model fails to retrieve the correct tool, the LLM never sees it, regardless of its reasoning capabilities. This creates a hard ceiling on system performance determined by the retriever's recall, making the choice of embedding model and similarity metric necessary design decisions.

The retrieval bottleneck is particularly problematic for rare or specialized tools. A tool that gets invoked infrequently will have few query examples in the embedding space, which makes it harder to surface through similarity search. This is a form of the long-tail problem: common tools become even more dominant because they are more easily retrieved, while niche but potentially valuable tools are systematically underexposed. Addressing this requires careful attention to description diversity and, in some cases, data augmentation techniques that generate synthetic query examples for underrepresented tools.

When you anticipate having dozens or hundreds of tools, description design requires a systematic approach rather than ad-hoc writing. Several principles improve description quality at scale.

Consistent vocabulary: Choose a consistent vocabulary for describing tool capabilities across your entire ecosystem. If some tools use "retrieve" and others use "fetch," "get," and "obtain" to describe the same operation, embedding similarity becomes noisy. Standardizing on a small set of action verbs that map to distinct capability categories helps embeddings cluster correctly.

Scope clarity: Each description should make the tool's scope unmistakably clear. Rather than "Handles user data," write "Read and write user profile fields including name and email plus preferences. Does not handle authentication or payment information." The more precisely you define what a tool covers, the sharper the decision boundary becomes.

Hierarchical structure: For large tool ecosystems, consider organizing tools into namespaces or categories. A tool named finance.get_balance is more clearly scoped than get_balance, and the namespace prefix helps the model reason about tool groups before selecting specific tools. This hierarchical organization also enables two-stage routing, where the first stage selects a category and the second selects a specific tool within that category.

Version and deprecation notes: In evolving systems, descriptions should note when tools are preferred over others. "Use this instead of the deprecated old_search_tool" directly steers the model away from legacy endpoints that might still appear in its training data.