Evidence record EV-0023

Hiding tools is not enforcing permissions

Across 2,160 attempts with four frontier models (preprint), a server with only in-body permission checks exposed forbidden tools in 152 of 720 trials and permission-aware visibility cut that to 0 of 720, yet models named a hidden tool in up to 94% of settings when it was inferable from the prompt.

Preprint · Measurement · Retrieved

Evidence class: Preprint. Unreviewed. Many are by one author or a small team, and some authors have a stake in the result. Most AX research is in this class today. Pattern tags: auth-scopes, dynamic-tools, discovery.

Effect, as the source reports it

  • Trials in which forbidden tools were exposed. Baseline: In-body checks only: 152 of 720 (21.1%). With the change: Permission-aware visibility: 0 of 720. Direction: decrease. Size: 21.1% to 0%. Sample: 2,160 attempts; four frontier models.
  • Settings in which models referenced a hidden tool by name. With the change: Up to 94% when the tool was inferable from the prompt. Direction: not-applicable. Size: Up to 94%. Sample: As above.

Agent profile

  • Note on models: Four frontier LLMs. The abstract does not name them.

Conflicts of interest

None declared in the abstract. The full text was not checked for a competing-interest statement.

Source

Zero-Trust Authorization and Discovery for Enterprise MCP, arXiv, Huan Li, Yuwei Wang, Srinivasan Manoharan, 18 September 2026, arXiv:2609.22573. Retrieved ; verification: abstract-only.

Every number in this record was checked against the live arXiv abstract page on 2026-10-08. The full text was not re-checked.

Limitations

  • Preprint, not peer reviewed.
  • Visibility-only filtering remained bypassable by scripted clients.

For designers

Hiding the tools a caller may not use makes the catalogue easier to choose from, but it is not a security boundary. Enforce permissions when a tool is called.

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0023" and this page's address, https://agentexperience.tech/evidence/ev-0023/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.