Evidence record EV-0040

Agent skills recommended a package that does not exist

Merged pull requests in Vercel's agent plugin repository corrected skill instructions that recommended an npm package that is not published, and plugin guidance that advertised deployment cards the production MCP server does not expose.

Vendor measurement · Observation · Retrieved

Evidence class: Vendor measurement. The vendor chose the tasks, ran the evaluation and wrote it up. Nobody else has reproduced it unless the record says so. Pattern tags: drift, skills.

Effect, as the source reports it

  • Facts in agent instructions that did not match production. With the change: A recommended package (@neondatabase/vercel-postgres-compat) returns E404 on npm; AI SDK version facts were wrong; plugin instructions advertised interactive deployment cards production MCP does not expose and described the connection as read-only. Direction: not-applicable. Size: Several drifted facts corrected across two pull requests. Sample: Two merged pull requests (#306, #307), following a run of drift pull requests dated 2026-10-05.

Agent profile

  • Note on models: Not an agent run: corrections to agent-facing instruction files.

Conflicts of interest

Vercel's own pull requests about its own plugin; recorded as an example of drift, not a criticism.

Source

vercel/vercel-plugin#306: Fix the leftovers the 2026-10-05 drift PRs corrected elsewhere, GitHub, molebox, 5 October 2026. Retrieved ; verification: verified-live.

The full primary source was fetched on 2026-10-08 and every number in this record was found in it. Both pull requests were merged (2026-10-05 and 2026-10-06). `npm view @neondatabase/vercel-postgres-compat` returned E404 on the retrieval date.

Corroborating sources:

Limitations

  • One vendor's repository; shows that drift happens, not how often.

For designers

Skills, plugins and llms.txt files go out of date as products change. Check mechanically that every package, tool and command they name exists in production.

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0040" and this page's address, https://agentexperience.tech/evidence/ev-0040/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.