What is a tool router in an AI agent platform?

The piece of infrastructure that decides which tool definitions actually reach the model on a given call. A platform might expose thousands of integrations, but a model only ever sees the handful the router decided were relevant — everything else stays out of the context window entirely.

Why can’t you just give the model every tool at once?

Two real costs. Token cost: ten thousand tool schemas at roughly 586 tokens each — the measured average of four real tool schemas — is about 5.86 million tokens, several times over even a 1-million-token context window, before the conversation starts. Accuracy cost: Anthropic’s own documentation states plainly that "Claude’s ability to pick the right tool degrades once you exceed 30–50 available tools" — more options in front of the model make it more likely to pick the wrong one, not less.

What does recall@k mean for tool routing?

Of the k tools a router hands back for a given request, how many of them are actually tools the task needed. A router with high recall@5 usually surfaces the right tool somewhere in its top five candidates; a router with low recall@5 makes the model choose from options that don’t include the one it actually needed, no matter how good the model is.

How does Anthropic’s tool search feature actually work?

You mark most tool definitions defer_loading: true and include a small tool-search tool (a regex or a natural-language BM25 variant). Claude searches the catalog, the API returns up to five matching tool_reference blocks by default, and only those get expanded into full definitions in context — everything else stays out. Anthropic reports this typically cuts a multi-server tool-definition footprint by over 85 percent.

How do you evaluate whether a tool router is any good?

Build a labeled test set of real requests paired with the tool each one should resolve to, measure recall@k on that set, and then check the end-to-end task success gap — the difference between the router picking the right tool and the agent actually completing the task with it. A router can score perfectly on recall and the agent can still fail downstream, and the reverse also happens.

The Tool Router: the Right 5 Tools out of 10,000

An agent platform can wire up thousands of tools; one model call has one context window. We measure what a tool actually costs in tokens, price what ten thousand of them costs per call, and walk five ways platforms narrow the catalog down to the handful that matter — plus how to tell whether the narrowing actually worked.

Concept · AI / ML. The source ↗

A free, interactive, animated visual explainer of The Tool Router: the Right 5 Tools out of 10,000 — built to be understood, not skimmed.

Questions

What is a tool router in an AI agent platform?: The piece of infrastructure that decides which tool definitions actually reach the model on a given call. A platform might expose thousands of integrations, but a model only ever sees the handful the router decided were relevant — everything else stays out of the context window entirely.
Why can’t you just give the model every tool at once?: Two real costs. Token cost: ten thousand tool schemas at roughly 586 tokens each — the measured average of four real tool schemas — is about 5.86 million tokens, several times over even a 1-million-token context window, before the conversation starts. Accuracy cost: Anthropic’s own documentation states plainly that "Claude’s ability to pick the right tool degrades once you exceed 30–50 available tools" — more options in front of the model make it more likely to pick the wrong one, not less.
What does recall@k mean for tool routing?: Of the k tools a router hands back for a given request, how many of them are actually tools the task needed. A router with high recall@5 usually surfaces the right tool somewhere in its top five candidates; a router with low recall@5 makes the model choose from options that don’t include the one it actually needed, no matter how good the model is.
How does Anthropic’s tool search feature actually work?: You mark most tool definitions defer_loading: true and include a small tool-search tool (a regex or a natural-language BM25 variant). Claude searches the catalog, the API returns up to five matching tool_reference blocks by default, and only those get expanded into full definitions in context — everything else stays out. Anthropic reports this typically cuts a multi-server tool-definition footprint by over 85 percent.
How do you evaluate whether a tool router is any good?: Build a labeled test set of real requests paired with the tool each one should resolve to, measure recall@k on that set, and then check the end-to-end task success gap — the difference between the router picking the right tool and the agent actually completing the task with it. A router can score perfectly on recall and the agent can still fail downstream, and the reverse also happens.

The Tool Router: the Right 5 Tools out of 10,000

Questions

Related explainers