Evidence record EV-0021

Agents leave available tools unused: the adoption gap

On OSWorld-MCP (preprint), a reasoning model given MCP tools called one on only 55 of 309 tasks, 23.9% of the tasks a tool could reach, and the same tools made a non-reasoning model 5.9 points worse.

Preprint · Measurement · Retrieved

Evidence class: Preprint. Unreviewed. Many are by one author or a small team, and some authors have a stake in the result. Most AX research is in this class today. Pattern tags: propensity, selection.

Effect, as the source reports it

  • Task success with MCP tools added to a GUI agent. Baseline: GUI only. With the change: GUI plus MCP tools: reasoning model +4.0 points; non-reasoning model -5.9 points. Direction: mixed. Size: +4.0 and -5.9 percentage points. Sample: 309 tasks; 5 runs each.
  • Tasks on which the reasoning model called a tool. With the change: 55 of 309 tasks. Direction: not-applicable. Size: 23.9% of tool-reachable tasks. Sample: 309 tasks.

Agent profile

  • Note on models: One reasoning and one non-reasoning model. The abstract does not name them.
  • Harness: One identical GUI-MCP harness on OSWorld-MCP.
  • Extensions: MCP tools from OSWorld-MCP.

Conflicts of interest

None declared in the abstract. The full text was not checked for a competing-interest statement.

Source

Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agents, arXiv, Siqi Fan, Minghao Li, Xiaoqian Ma, Wenhui Tan, Xiusheng Huang, Juntong Wu, Liujie Zhang, Shuo Shang, Weihang Chen, 4 August 2026 (v2, revised 2026-08-06), arXiv:2608.03327. Retrieved ; verification: abstract-only.

Every number in this record was checked against the live arXiv abstract page on 2026-10-08. The full text was not re-checked.

Limitations

  • Preprint, not peer reviewed.
  • One benchmark.
  • Training that raised tool adoption did not raise held-out accuracy.

For designers

Shipping a tool does not mean agents will use it. Measure how often agents call it on the tasks it could serve, not only whether it works when called.

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0021" and this page's address, https://agentexperience.tech/evidence/ev-0021/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.