Tool Selection for LLM Agents: Routing Strategies and Implementation › Key Design Parameters

The key parameters for the hybrid tool router interact in non-obvious ways, and understanding them helps you tune the system for your specific requirements:

• embedding_model: The sentence transformer model used to encode tool descriptions and queries. all-MiniLM-L6-v2 provides a good balance of speed and accuracy, though domain-specific models may offer better performance for specialized tool ecosystems. Larger models like all-mpnet-base-v2 provide better accuracy at higher latency.
• top_k: Number of candidate tools to retrieve via embedding similarity. Higher values give the LLM more options but increase context length and latency. Typical values range from 3 to 10, depending on the semantic similarity between tools in your ecosystem. For ecosystems with many similar tools, larger $k$ reduces the risk of the retriever missing the correct tool.
• threshold: Minimum similarity score for routing. Below this threshold, no tool is selected, triggering a fallback response. This parameter acts as a confidence gate, preventing the system from invoking tools when user intent is unclear or unrelated to available capabilities. Per-tool thresholds often work better than a single global value.
• tool_text_construction: The method used to build the text representation for each tool before embedding. Including examples in the embedded text ("calculator: Evaluate math expressions. Examples: What is 2+2? Calculate sqrt(16)") consistently outperforms embedding the description alone, because the examples introduce the query-like vocabulary that will appear in actual routing queries.