A paper dropped recently that should worry anyone building agent toolsets. The authors ran a stress test: give an LLM a set of tools and ask it to pick the right one. With five tools, it got 85 percent right. With twenty, that dropped to 45 percent. Add more tools past that, and the baseline collapses further.
The paper is called RAG-MCP, and it’s about solving the problem it documents. But the problem itself matters more than their fix.
The math is worse than it looks
Forty-five percent with twenty tools. That sounds bad. The real issue is what happens when you add tool number twenty-one.
Each new tool doesn’t just add one more option to choose from. It changes the decision surface for every existing tool. The agent now has to distinguish the right tool from a larger set of plausible alternatives. The noise grows faster than the signal.
This is not a context window problem. It’s a structural one. Even if you could fit ten thousand tool descriptions in the prompt, the model would get worse at picking, not better. More information, less accuracy. That’s the paradox.
Why this happens
Agents don’t have a tool catalog they consult. They have a prompt with every tool description stuffed into it, and they have to pattern-match the user’s query against all those descriptions simultaneously. When the set is small, the signal is clear. When it grows, three things degrade at once.
First, attention dilution. The model’s attention gets spread across more schemas. Each tool competes for the same finite attention budget.
Second, semantic crowding. More tools means more similar tools. A file reader and a document reader and a spreadsheet reader start to look like the same thing in embedding space. The model confuses cousins.
Third, instruction interference. Tool descriptions carry their own behavioral hints. Conflicting instructions from different tools create internal noise that degrades the model’s ability to follow any single tool’s contract.
The result is what I call the simplicity tax. You pay it every time you add a capability. The tax compounds because it hits every existing tool, not just the new one.
The fix everyone reaches for doesn’t work
Most teams respond by building better prompts. Longer descriptions, more examples, clearer formatting. This helps marginally and costs tokens linearly. At some point, you hit the wall where adding clarification to the prompt makes it longer, which makes attention dilution worse, which requires more clarification. It’s a feedback loop.
Some teams respond by restricting the toolset. Only give the agent the tools it needs for this task. That works until the task is underspecified, which is the whole reason you built an agent in the first place. If you knew which tools were needed, you wouldn’t need the agent.
What RAG-MCP gets right and wrong
The paper’s approach is to retrieve relevant tools before showing them to the model. Instead of dumping all tool descriptions into the prompt, a retriever picks the top candidates and only those get injected. This cuts prompt size and improves accuracy.
That’s a good engineering solution to a bad architectural assumption. The assumption it doesn’t challenge: that the agent should face a toolbox at all.
The retrieval layer becomes a new failure surface. If the retriever misses the right tool, the agent never sees it. You’ve traded decision overload for retrieval blind spots. The paper acknowledges this: retrieval precision degrades as the tool registry scales to thousands. The problem didn’t go away. It moved upstream.
The deeper issue
The root problem is that we’re asking agents to reason about their own capabilities while using those capabilities. It’s like a mechanic who has to read every tool manual before picking up a wrench. The manual-reading is not the job. Using the wrench is.
A better architecture would separate tool discovery from tool execution. The agent should not browse its own toolbox. It should declare what it needs, and a separate layer should map that declaration to an implementation. This is the difference between “pick a tool” and “describe a need.” One is a search problem. The other is a specification problem.
Specification is harder to get right upfront but scales without degradation. You add a new tool, update the mapping layer, and the agent’s existing behavior is unaffected. No attention tax. No semantic crowding. The agent never sees tools it won’t use.
The uncomfortable conclusion
Most agent frameworks are building in the wrong direction. They’re making toolsets larger and prompts smarter, when the real solution is to make tool selection someone else’s problem. Not the agent’s.
Every tool you add to an agent’s prompt is a capability you’re giving it and a cognitive cost you’re imposing on it. The cost is not proportional to the number of tools. It compounds. The twentieth tool costs more than the first.
Build agents that know what they need, not agents that know everything they have.
The difference is the difference between a craftsman who reaches for the right tool without thinking and one who has to read the label on every drawer first.