Evidence record EV-0032

Vendor claim: agents never invoked a docs skill in 56% of eval cases

Vercel states that in its Next.js 16 evals the docs skill was never invoked in 56% of cases, so the skill matched the 53% pass rate of no docs, while an 8KB docs index in AGENTS.md reached 100%.

Vendor claim · Vendor claim · Retrieved

Evidence class: Vendor claim. A claim, not a measurement. The definitions behind it are usually unstated, and the figure often comes from a launch or funding announcement. It can show a direction, not a size. Pattern tags: propensity, skills, agents-md, docs-for-agents.

Effect, as the source reports it

  • Eval cases in which the skill was never invoked. With the change: 56%. Direction: not-applicable. Size: 56%. Sample: Not stated.
  • Pass rate on the hardened Next.js 16 eval suite. Baseline: No docs: 53%. With the change: Skill, default: 53%; skill with explicit AGENTS.md instructions: 79%; docs index in AGENTS.md: 100%. Direction: increase. Size: +0, +26 and +47 percentage points. Sample: Not stated.
  • Skill trigger rate with explicit instructions. Baseline: Default: skill not invoked in 56% of cases. With the change: Explicit instruction in AGENTS.md: 95%+. Direction: increase. Size: To 95%+. Sample: Not stated.

Agent profile

  • Note on models: The post does not name the models or the agent used.

Conflicts of interest

Vercel maintains Next.js and the @next/codemod tool that installs the AGENTS.md docs index the post recommends.

Source

AGENTS.md outperforms skills in our agent evals, Vercel, Jude Gao, 27 January 2026. Retrieved ; verification: verified-live.

The full primary source was fetched on 2026-10-08 and every number in this record was found in it.

Limitations

  • A vendor claim: no run counts, model names or raw data are published.
  • The vendor's own eval suite, focused on Next.js 16 APIs not in model training data.

For designers

Do not assume an agent will choose to load a skill. For knowledge every task needs, passive context such as an AGENTS.md entry was more reliable in this vendor's tests; check it on your own tasks.

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0032" and this page's address, https://agentexperience.tech/evidence/ev-0032/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.