Keep agent instructions true.

Short answer. Agent instructions name concrete things: packages to install, tools to call, commands and flags to run, and URLs to read. Those names go out of date, and an agent acts on a stale name with full confidence. Treat every name as a claim that a machine can check. Resolve each one in CI against the package registry, the live server, the released CLI or the web, and fail the build when one does not resolve. Then give each file an owner and a review rhythm.

Where agent instructions live

Agent-facing instructions now sit in several places, and each one names things.

The MCP specification describes that last field as optional “guidance for LLMs on how to use this server effectively”. The agents.md site says to “treat AGENTS.md as living documentation”. Neither can check that the guidance is still true. That job is yours.

What drift looks like in public

Several public fixes from the last few weeks show the shape of the problem.

• A package that is not published. In vercel/vercel-plugin #306 (merged 5 October 2026), Vercel fixed a skill that pointed agents to @neondatabase/vercel-postgres-compat. The pull request notes that the package is not published: npm view returned E404. The same change corrected AI SDK version facts, such as which major version removed or deprecated an option.
• Features production does not expose. vercel/vercel-plugin #307 (merged 6 October 2026) states: “Plugin instructions advertise interactive deployment cards that production MCP does not expose and describe the connection as read-only.” It also replaced an outdated tool name.
• Configuration keys the platform rejects. vercel/vercel-plugin #252 (merged 25 September 2026) added a check after a skill taught a vercel.json key Vercel does not accept: “Skills can teach vercel.json keys that Vercel does not accept, and nothing catches it.”
• Guidance that contradicts the CLI’s own defaults. neondatabase/agent-skills #135 (opened 3 October 2026) aligned a skill with the Neon CLI after “an agent following that guidance flagged the default as wrong”. It also removed a flag that the current CLI rejects.
• A safety claim the tool did not meet. In neondatabase/mcp-server-neon #367 (merged 30 September 2026), an explain tool defaulted to EXPLAIN ANALYZE, which runs the statement, while its annotations “described every call as read-only, non-destructive, and idempotent”. The fix changed the default and the annotations.

None of this is a criticism of these teams. They found the drift and fixed it in public, which is exactly what this guide recommends. Vercel’s resync pull requests are opened by what the repository calls a “drift agent”, in batches a few days apart, and each one carries a note asking for review before merge.

Why drift matters more for agents

A person who reads a stale instruction often notices: the package will not install, the page looks old, a colleague mentions the rename. An agent has none of those cues. It has the instruction, and it follows it.

Research on repository context files supports the “it follows it” part. Gloaguen et al., “Evaluating AGENTS.md” (preprint, first posted February 2026, revised September 2026) tested several coding agents and models on SWE-bench tasks and a new set of real repository issues. They report that agents do follow the instructions in such files, while the files did not generally improve task success and raised inference cost by over 20% on average. The abstract does not name the models, so read the result as dated to the systems tested. A file that is followed but wrong is worse than no file.

A missing package name is also a security problem. Spracklen et al. studied package names invented by code-generating models (arXiv 2406.10279, USENIX Security 2025). Across 16 models, two programming languages and 576,000 code samples, they report that on average at least 5.2% of the packages suggested by commercial models, and 21.7% of those suggested by open-source models, did not exist, with over 205,000 unique invented names. An attacker can publish a package under such a name and wait. This is often called slopsquatting. Earlier, in March 2024, Lasso Security described publishing an empty package under a name models kept inventing, huggingface-cli, which it says received more than 30,000 downloads in three months (Lanyado, 2024).

Skills spread those names. In January 2026, Aikido reported an invented npx command that had been copied into skill files in at least 237 GitHub repositories (Aikido, 2026), and a public issue on one large skill collection reported skills “using non-existent npm packages, vulnerable to slopsquatting attacks” (wshobson/agents #424). A skill is code that runs through someone else’s agent. A package name in it deserves the same scrutiny as a dependency in your lockfile.

How to check: resolve every name in CI

Every name in an instruction file has a place where it either exists or does not. A drift check finds each name, resolves it there, and fails the build when it does not resolve.

1. Extract the names. Pull package names from install commands and imports in code blocks, tool names from text and manifests, commands and flags from shell blocks, and every link. Code blocks are the easy part; names in prose need a list you maintain.
2. Resolve packages in the registry. npm view <name> or the equivalent for your ecosystem. A 404 is a failure. So is a package that exists but whose owner you do not recognise, which is how a slopsquatted name would look.
3. Resolve tools against the live server. Call the MCP server’s tool listing and compare it with every tool name the instructions mention, including in MCP server instructions themselves.
4. Resolve commands and flags against the released CLI. Install the version users get, not the one on your branch, and parse its --help.
5. Fetch every URL. Expect a success status. Treat a redirect to a generic page as a failure, because a moved page usually means a renamed thing.
6. Test behaviour claims separately. “Read-only”, “safe to retry” and “does not deploy” cannot be proved by a name lookup. Write a test that calls the tool against a sandbox and checks that nothing changed. That is the kind of test that would catch the Neon case above.

Existing tools cover parts of this. Claude Code’s claude plugin validate checks plugin and skill files for syntax and schema errors, with exit codes CI can act on; its documentation is clear that it checks files, not behaviour. Open-source linters for AGENTS.md and similar files are appearing, and some check that the paths and scripts a file mentions exist in the repository. This site has a small example of its own: a unit test fails if the tool names in its MCP catalogue and in its /.well-known/ard.json disagree. An open-source drift check for agent instruction files is in development here.

Who owns the file

Drift lasts because instruction files sit between teams. The docs team writes llms.txt, a developer relations engineer writes the skill, the platform team ships the MCP server, and nobody owns the gap between them.

Name one owner per file, in a CODEOWNERS entry or the file’s front matter. The owner does not have to write every line. They have to be the person the failing check asks. When the product team renames a tool, removing it from the MCP server should fail the instruction check in the same pull request, so the person making the change fixes the instruction too.

How often to review

Run the automated check on two triggers.

• On every change to the instruction file or to anything it names that lives in your repository: the CLI, the server, the docs.
• On a schedule for things you do not control: third-party packages, external documentation and other teams’ servers. Nightly is cheap. A package that existed yesterday can be unpublished today.

A person should also read each file whenever the product changes shape, not only when a check fails. Checks catch names that stopped existing. They do not catch advice that is still valid but no longer the best route, such as a skill that still teaches an API the vendor now calls deprecated.

A worked example

The product and every name here are invented for illustration. Larkspur is a hosting platform with a CLI, an MCP server and a skill. Its skill says:

Install the client with `npm install @larkspur/sdk-compat`.
Deploy a preview with `larkspur deploy --preview`.
Read build errors with the `get_build_logs` MCP tool.
Full reference: https://docs.larkspur.example/cli/deploy

The nightly drift check produces this report:

package  @larkspur/sdk-compat          FAIL  npm: E404, not published
flag     larkspur deploy --preview     FAIL  larkspur 3.0.1: unknown option (renamed to --target preview)
tool     get_build_logs                FAIL  live server lists get_deployment_logs
url      /cli/deploy                   PASS  200

Three of four names fail. Without the check, an agent following the skill would try to install a package that does not exist, which is exactly the name an attacker would want to register. It would then pass a flag the CLI rejects and call a tool the server no longer has. Each failure would cost the user a round of confusion, and if anyone had registered the missing package name, the first one would install their code.

The fix takes ten minutes: correct the three names and add the skill’s owner to the failure notification. The lasting fix is that the tool rename in the MCP server now fails the skill check in the same pull request.

A checklist

1. Do you know every file that gives agents instructions about your product, including skills and plugins published by your team elsewhere?
2. Does each file have one named owner?
3. Does CI resolve every package name in the registry, and fail on a 404?
4. Does CI compare tool names against the live server’s tool listing?
5. Does CI check commands and flags against the released CLI, not your branch?
6. Does CI fetch every URL?
7. Is every behaviour claim, such as read-only, covered by a behaviour test?
8. Does a scheduled run catch changes in things you do not control?

In the Open Agent-Readiness Rubric, this is check READ-11: instructions written for agents agree with production, so every command, package, tool name, flag and link they name exists and does what they say. The checklist above is one way to meet it.

For the description side of the same problem, see Write tool descriptions an agent can act on. For large surfaces, where instructions often name commands and flags, see How to expose a 3,000-operation API to agents.

Frequently asked questions

What is drift in agent instructions? Drift is the gap between what an agent-facing file says and what production does. A skill names a package that is not published, an AGENTS.md file gives a flag the CLI no longer accepts, or MCP server instructions describe a tool that the live server does not expose. The file still reads well, so nobody notices until an agent acts on it.

What are AGENTS.md best practices? Keep it short and specific to the repository: build and test commands, code style, hazards and house rules. The agents.md site describes it as a README for agents and says to treat it as living documentation. Every command, path and package it names should be something a CI job can run or resolve, so the file cannot quietly go out of date.

How do I keep llms.txt up to date? Generate it from the same source as the pages it links to, rather than writing it by hand, and check every link in CI. If you write it by hand, fetch each URL on every deploy and fail the build on a 404 or a redirect to an unrelated page. The llmstxt.org proposal also suggests testing the file by asking an agent questions about your content.

What is skill drift? Skill drift is when a skill’s instructions fall behind the product or library it teaches. The skill keeps recommending an old API, a removed option or a package name that never existed, and agents that load the skill copy the mistake into new code. Vercel’s plugin repository fixes this with recurring pull requests that resync each skill against current documentation.

What is slopsquatting? Slopsquatting is registering a package under a name that AI models tend to invent, so that code or instructions that use the invented name install the attacker’s package. It matters for agent instructions because a skill or AGENTS.md file that names a package which does not exist gives an attacker a ready-made target.

How often should agent instructions be reviewed? On every release of anything they name, and on a schedule for things you do not control, such as third-party packages and external documentation. An automated check can run on every pull request and every night; a person should read the file whenever the product it describes changes shape.

An instruction file is finished when every name in it resolves, and stays finished only while something checks that it still does.