Progressive MCP Tool Routing: How We Cut Agent Hallucination by 40% in a 47-Tool Environment
The Problem: Agent Cognitive Overload in Complex Toolsets
As AI agents graduate from simple assistants to autonomous developers, their toolsets are scaling dramatically. We recently audited a production-grade agent system with access to 47 distinct Model Context Protocol (MCP) tools—spanning code execution, database queries, cloud infrastructure management, and real-time data analysis. The results were alarming.
The system was loading the full tool manifest—52,000 tokens—into the agent's context window for every single decision. This wasn't just expensive; it was catastrophic for performance. Agents spent 70% of their reasoning budget parsing tool definitions instead of executing tasks. Worse, we documented a 22% hallucination rate where agents would confidently call non-existent tools or misinterpret parameter schemas, leading to runtime failures. The core issue was clear: forcing an agent to comprehend the entire MCP ecosystem at once was unsustainable.
Case Study Baseline: The 47-Tool MCP Monolith
Let's examine the specific architecture we evaluated. The MCP server exposed a flat namespace of tools, each with a detailed JSON schema. A sample tool definition for a database query function looked like this:
{
"name": "query_postgres_analytics",
"description": "Executes read-only SQL queries against the production analytics replica.",
"parameters": {
"type": "object",
"properties": {
"query_string": { "type": "string", "description": "Valid SELECT statement. CTEs are permitted." },
"result_format": { "enum": ["json", "csv"], "default": "json" },
"timeout_seconds": { "type": "integer", "default": 30 }
},
"required": ["query_string"]
}
}With 47 such tools, the agent's context was dominated by schema boilerplate. We measured average token usage per task cycle at 52,400 tokens, with a peak of 58,100 tokens. More critically, the agent's "tool selection accuracy"—measured by correct first-call tool invocations—was only 68%. The remainder involved error handling, retries, and corrective reasoning that ballooned the token count.
Implementing Progressive Disclosure with Semantic Tool Search
Our solution replaced the monolithic tool manifest with a three-phase progressive routing strategy. The key innovation was decoupling tool discovery from tool execution using a vector-based semantic index.
Phase 1: Intent Resolution & Tool Routing. The agent receives a high-level task. Instead of seeing 47 tools, it accesses a compact, 3,000-token "router guide" that describes 5 primary tool categories (e.g., "Data Analysis," "Infrastructure Control," "Code Generation"). The agent first selects a category.
Phase 2: Semantic Search Within Category. Upon category selection, the system performs a semantic search against a pre-embedded vector store of tools within that category. For a query like "summarize last quarter's user growth by region," the search retrieves the top 3 relevant tools from the Data Analysis set, returning only their core schemas. This is the semantic tool search in action—it uses embeddings of the tool descriptions and parameter schemas to find the best matches, not just keyword matching.
# Simplified view of the semantic router
import pinecone
from sentence_transformers import SentenceEncoder
index = pinecone.Index("mcp-tool-vectors")
encoder = SentenceEncoder("all-MiniLM-L6-v2")
def find_tools(user_task, top_k=3):
task_embedding = encoder.encode(user_task)
results = index.query(vector=task_embedding, top_k=top_k, filter={"category": "data_analysis"})
# Returns simplified schemas, not the full 1000+ token definition
return [load_brief_schema(tool_id) for tool_id in results['matches']]Phase 3: On-Demand Schema Disclosure. Only once the agent selects a specific tool for execution does the system inject the full, detailed schema—including examples, edge cases, and precise type constraints—into the context. This is true progressive disclosure, ensuring the agent only pays the token cost for tool complexity when it's absolutely necessary.
The Results: 40% Fewer Hallucinations and 65% Token Savings
We deployed this progressive routing architecture and ran the same suite of 200 benchmark tasks against the original system. The improvements were transformative:
- Hallucination Reduction: Incorrect tool calls dropped from 22% to 13.2%, a 40% relative reduction. The agent stopped inventing tools that didn't exist because it was only considering a curated, relevant subset.
- Token Efficiency: Average token consumption per task plummeted from 52,400 to 18,340 tokens, a 65% savings. This directly translates to lower latency and inference costs.
- First-Call Success: Tool selection accuracy on the first attempt jumped from 68% to 89%. The semantic search provided highly relevant options, reducing the trial-and-error loop.
- Scalability: The agent's performance remained stable when we added 15 more tools to the MCP server. The progressive system scales without degrading agent reasoning.
Implementation Blueprint: Agent Context Optimization
Adopting this pattern requires rethinking your MCP architecture. The core components are: a tool vector index, a category-based routing manifest, and a schema loader that can fetch and format schemas on demand. Your agent prompt must be rewritten to guide it through the multi-step selection process. We use a system prompt that explicitly states: "You first select a domain, then search for tools within that domain, then execute."
This is a fundamental shift towards agent context optimization. You are managing the agent's cognitive load deliberately. The investment in setting up the vector index and the routing logic pays dividends in reliability, cost, and speed. It turns an unwieldy toolset from a liability into a scalable asset.
Ready to implement progressive routing and tame your complex toolsets? Explore the architecture patterns and start optimizing your agent's context at TormentNexus.site.