How Velaris picks 1 tool from 1,000: the capability mesh

AI agent tool selection is the problem of choosing the right tool for a step from everything an agent could theoretically use. It sounds trivial with 10 tools and becomes the central engineering problem at 1,000. Here’s how we solved it: connectors are data an agent searches, not definitions we cram into a prompt.

The wall you hit at about 40 tools

The standard approach is to pass every tool definition in the request. It works beautifully in demos. Then you add tools and three things break at once:

1. Context cost. A rich tool schema is 200–800 tokens. A thousand of them is a few hundred thousand tokens — before the user has said anything. You’re paying for a catalog on every single call.
2. Accuracy collapse. Selection accuracy degrades as the candidate set grows. With 40 near-identical CRM actions in front of it, a model picks a plausible one — not necessarily the right one.
3. Latency. Bigger prompts, slower first token, on every request.

You can’t fix this by prompt-engineering harder. The catalog is the problem.

Connectors as data

So we stopped treating tools as part of the prompt and started treating them as part of a searchable index. Every connector — Gmail, Slack, a Postgres database, an internal endpoint — is a row: what it does, what it needs, what it costs, what tier it is, which scopes it requires.

An agent step then looks like this:

1. Search. The planner describes the step in natural language (“find the customer’s last three invoices”). We retrieve candidate capabilities against that intent — hybrid lexical + semantic, filtered by what this user has actually connected.
2. Rank. Candidates get scored on intent match, the user’s connected accounts, historical success on similar steps, and cost/latency. A cheap exact tool beats an expensive general one.
3. Bind. Only the shortlist gets materialized into the model’s context — summaries for the candidates, full schemas for what it actually calls — along with the scoped credentials for this run and nothing else. How long that shortlist should be turned out to be a measurable question, not a taste one; see below.
4. Execute and learn. Success and failure feed back into the ranking signal.

The model never sees a thousand tools. It sees the handful that could plausibly do this step. Selection accuracy goes up because the choice got easier, and the context cost stops scaling with the size of the catalog.

How big should that handful be? We used to say five. Then we measured it, and five is wrong. Across the same 232-case eval, the share of queries where the correct tool appears anywhere in the returned shortlist runs 93.1% at five candidates, 95.7% at eight, 98.3% at fifteen, and 99.6% at twenty. When we dug into the cases that failed at five, the right tool was sitting at a median rank of ten out of fifty-eight — the retrieval had found it and the window was too narrow to show it.

Twenty candidates costs about 280 extra tokens of summaries, against a measured median of 8,213 prompt tokens per agent step. So the shortlist should be wider than instinct suggests: retrieval’s job is to be nearly certain the answer is in the window, and the model’s job is to pick from it.

The cost of the alternative is measurable. Our agent runner binds its whole tool surface into every model step — measured over 93 steps in July 2026, that’s a median 8,213 prompt tokens per step (p90: 11,346) before the model reads a word of the actual task. The mesh path exists so that number stops growing with the catalogue.

What actually turned out to be hard

The retrieval isn’t the hard part — that’s a solved shape. The hard parts were the boring ones:

Description quality dominates everything. Retrieval is only as good as what the connector says about itself. Tool descriptions written by engineers (“execute query against configured datasource”) retrieve terribly against how users actually phrase intent. We rewrite every connector description in user language, and now we can quantify it. On a 232-case labelled eval covering all 58 published tools, the catalog described in API-speak scored 38% top-1; our worst category, team-chat, sat at 8%. Rewriting those six descriptions alone took the category to 83% — but pulled six queries from other categories toward it, because in a shared ranking pool description quality behaves like gravity. So we levelled everything: after rewriting all 51 remaining descriptions into what-it-does-plus-when-to-use-it user vocabulary, the same 232 cases score 64% top-1 / 93% top-5 (July 2026, all three runs recorded as Langfuse dataset runs). The number that proves the mechanism: colloquial phrasings — queries sharing almost no words with the tool’s name — went from 26% to 60% top-1. Description quality didn’t just move the numbers; it was most of the gap between keyword matching and actual semantic retrieval.

There is a limit to that, and we found it by overshooting. Most of the remaining errors were sibling confusions — a tool that operates on one item losing to the tool that operates on the whole container. So we wrote a third revision that stated the distinction explicitly and contrastively: the unit brought into existence is the individual to-do; preparing the named container it will sit in is a separate capability. Precise, unambiguous, and 7.8 points worse — the very family it targeted dropped from 36% to 25%. Formal, abstract prose reads well to a human and moves the text away from the everyday words a person actually types, which is the only thing the embedding is matching on. We simulated it against the eval before deploying, saw the regression, and threw it away. Concrete beats precise.

Near-duplicates need tie-breakers. “Send message” exists in Slack, Teams, Discord and Telegram. Intent alone won’t separate them; the user’s context does — what’s connected, what they used last time for this kind of step, what the surrounding thread mentions.

Retrieval cannot read the word “not.” This one took us the longest to even see, and it is worth naming. When someone writes “don’t put this on my task list — open it as a ticket in the repo,” the excluded surface is still sitting there in the sentence, and an embedding has no mechanism for negation: the vector for that request looks substantially like a request about task lists. In our eval this was the single largest failure class — seven of the sixteen queries where the right tool never appeared at all. The proof is clean: delete the negated clause and six of the seven retrieve correctly on the next try. No amount of description writing fixes it, because the defect is on the query side. The fix has to happen on the query, before it is ever embedded: detect the negated span and delete it. We built that, and the interesting part was getting it wrong first. The obvious version drops the whole negated clause, which lifted top-1 by 2.6 points and cost us recall — because the negated clause routinely carries the actual request inside it (“not in my issues and not in Drive, search the email messages”). Dropping the clause drops the ask. The version that works strips only the negative spans within a clause, and treats conversational negatives — “I can’t remember which list I used” — as the hedges they are rather than as exclusions. On the same 232 cases that takes retrieval to 69% top-1 / 95% top-5, with fifteen queries gaining the top slot, none losing their place in the shortlist, and every query containing no negation at all returned byte-identical — which is the property that actually matters, since that is the overwhelming majority of them.

Failure has to be legible. When binding fails, the agent must say “I couldn’t find a tool for that” rather than grabbing the nearest thing and improvising. Confidently wrong tool selection is the worst failure mode in the system, because it looks like it worked.

Every candidate needs a tier. Selection can’t be purely about capability, because “delete the records” and “read the records” are not interchangeable even when they retrieve similarly. Tier is a first-class ranking input, which is where this connects to the approval architecture.

Why this shape wins

There’s a version of this problem everyone will face. As agents take on more, the catalog grows, and any design where capability lives in the prompt has a ceiling baked in. Moving the catalog out — into an index the agent queries like a filesystem — means the thousand-and-first connector costs nothing extra at request time.

That’s the capability mesh: not a longer prompt, a better index. It’s also why adding a connector is a data operation for us rather than an engineering project.

Want to see it work? Get early access, or read why we built an OS around this in the first place.