Agent readiness
Score what an agent can actually see.
Agent readiness is not a feeling about a site. It is a set of surfaces that are either declared or not, and the declarations can be checked one at a time.
Agent readiness is checkable. Every criterion below resolves to a specific request, a specific piece of markup, or a specific response payload — something you can point at and settle in under a minute. This is version 0.1 of an open rubric: 36 criteria across six dimensions, weighted to a total of 100, published with its weights visible so you can disagree with them in the open.
The rubric is vendor-neutral and deliberately structural. It scores what a site or product declares. It does not score how well an agent performs, because that depends on the model, the harness, the task, and the day.
- Machine-readable version:
/agent-readiness-rubric.json— same criteria asid,dimension,criterion,check,weight. - Version: 0.1 (draft), 2026-09-01. Licence: CC BY 4.0.
What the rubric measures, and what it does not
The rubric has six dimensions. They follow the order in which an agent meets a product: it has to find you, parse you, read you, act, fail safely, and know what it is allowed to do.
| # | Dimension | Weight | The question it answers |
|---|---|---|---|
| 1 | Discovery | 20 | Can an agent find what exists here without guessing? |
| 2 | Structure and semantics | 18 | Does the machine-readable structure match what a person sees? |
| 3 | Machine-readable content | 17 | Is the substance available in a form a machine can consume exactly? |
| 4 | Action safety | 18 | Are consequences declared and guardrails enforced by the system, not the prompt? |
| 5 | Recovery | 15 | When something fails, is there a basis for a different second attempt? |
| 6 | Policy signals | 12 | Has the organisation said, machine-readably, what AI systems may do? |
Three things this rubric deliberately does not do. It does not rank vendors or tools. It does not claim that a higher score produces more traffic, more citations, better rankings, or more completed agent tasks — no public dataset we are aware of supports that link, and we will not assert it. And it does not treat the presence of a file as a success; several criteria explicitly test whether the file is current, because a stale index is worse than no index.
How the scoring model works
Each criterion is scored on a three-point scale and multiplied by its weight.
- 0 — absent. The criterion is not met.
- 0.5 — partial. Met for part of the surface, or met in a way an agent cannot rely on.
- 1 — met. Met across the surfaces an agent would reasonably encounter.
criterion score = value × weight. The total is the sum; weights add to 100. Report the six dimension subscores alongside the total — a site with everything in Discovery and nothing in Action safety is a different site from one scoring the same total evenly, and the total alone hides that.
Not applicable. Mark a criterion N/A only when the surface genuinely does not exist — action-safety criteria on a read-only reference site, for instance. Renormalise the remaining weights to 100 and publish which criteria you excluded.
Descriptive bands. These name what an agent can do with the site. They are descriptions, not grades, and they carry no claim about outcomes.
| Score | Band | What it describes |
|---|---|---|
| 0–24 | Unmapped | An agent has little to work with beyond rendered HTML. |
| 25–49 | Readable | Content is findable and parseable; actions and policy are undeclared. |
| 50–74 | Navigable | Discovery, structure, and content are deliberate; action and recovery are partial. |
| 75–100 | Actionable | Agents can find, read, act, fail safely, and be told what is permitted. |
Always publish the assessment date and method with the score. Readiness drifts.
Dimension 1 — Discovery (weight 20)
Can an agent find the content and capabilities that exist here, without guessing?
DIS-01 — A parseable robots.txt exists · weight 3
Check: GET /robots.txt. Expect 200 and text/plain. Confirm it is not an HTML error page served with a 200 status.
Why it matters: robots.txt is the oldest and most widely honoured machine-readable statement a site makes. A soft-404 returned as a 200 is worse than a clean 404, because parsers treat the HTML as a rule set and get nonsense.
DIS-02 — An XML sitemap exists and is referenced from robots.txt · weight 3
Check: GET /sitemap.xml, or the path named on the Sitemap: line. Confirm the line exists in robots.txt and that a sample of listed URLs return 200.
Why it matters: a sitemap is the cheapest complete inventory a site can publish, and both crawlers and retrieval agents use it to avoid walking the link graph. It is also the single most commonly missing file on otherwise well-built sites.
DIS-03 — Directives for AI and agent user-agents are explicit and intentional · weight 3
Check: read robots.txt for named AI user-agents — GPTBot, OAI-SearchBot, ClaudeBot, Google-Extended, PerplexityBot and their peers. Confirm the allow/deny pattern matches what the organisation actually intends.
Why it matters: training crawlers and answer-engine fetchers are different user-agents with different consequences, and sites frequently block the second while intending only to block the first. OpenAI’s publisher documentation is explicit that OAI-SearchBot should not be blocked if you want your pages to be summarised in ChatGPT search (OpenAI Publishers and Developers FAQ).
DIS-04 — A site-level agent index exists and is current · weight 3
Check: GET /llms.txt, or whichever index the site has chosen. Confirm 200, confirm the listed URLs resolve, and confirm the file was updated in the same release cycle as the content it points at.
Why it matters: the value here is a maintained, correct inventory — not the filename. Consumption of llms.txt by major assistants is contested, and Ahrefs’ June 2026 study of 137,210 domains reported that 97% of the llms.txt files it found received no traffic at all in May 2026 (Ahrefs, 15 June 2026). Score this criterion on correctness and maintenance rather than presence — a file listing URLs you deleted six months ago actively wastes an agent’s budget. See the standards matrix for the fuller evidence picture.
DIS-05 — Capabilities are declared in a machine-readable manifest at a well-known path · weight 4
Check: look under /.well-known/ for a capability manifest — an ARD ard.json, an A2A agent card, or an MCP server description. Validate it against its published schema and confirm the endpoints it names are live.
Why it matters: a content index tells an agent what to read; a capability manifest tells it what it can do. This is the rubric’s single heaviest criterion because it is the least substitutable: nothing else on a site answers the question “what actions does this product expose to me?”
DIS-06 — URLs are canonical, stable, and predictable · weight 2
Check: confirm rel=canonical on templated pages, no session identifiers in URLs, consistent trailing-slash behaviour, and 301 rather than soft redirects on moves.
Why it matters: an agent that revisits a URL from an earlier step must land on the same resource. Unstable URLs break multi-step tasks and invalidate every cached plan.
DIS-07 — Discovery surfaces cross-reference each other · weight 2
Check: confirm the sitemap, the agent index, the capability manifest, and the API reference each link to the others.
Why it matters: agents arrive at one entry point, not all of them. Cross-links mean any single entry point leads to the rest.
Dimension 2 — Structure and semantics (weight 18)
Does the page’s machine-readable structure match what a person sees?
STR-01 — Interactive controls use native semantic elements · weight 4
Check: inspect the accessibility tree along the primary task path. Every control that navigates should be a link; every control that acts should be a button; form fields should be real form fields.
Why it matters: browser agents read the accessibility tree. A div with a click handler has no role, no accessible name, and no keyboard contract — so for that reading, the control is not there. This is also why accessibility work and agent-readiness work overlap so heavily: they consume the same tree. Google’s own guidance for developers builds on exactly this point (web.dev, “Build agent-friendly websites”).
STR-02 — Every control has a meaningful accessible name · weight 3
Check: enumerate controls on the primary path. Confirm none resolve to an empty name, a bare icon glyph, or a generic label such as “Submit” repeated across different actions.
Why it matters: an agent selects an action by its name. Duplicate or empty names force selection by position, which breaks on the next layout change.
STR-03 — Heading hierarchy describes the content beneath it · weight 2
Check: extract the heading outline. Confirm a single h1, no levels skipped for styling reasons, and headings that name their section’s subject rather than carrying a slogan.
Why it matters: heading outlines are the cheapest chunking signal available to a retrieval system, and they are how an agent decides which part of a long page to read.
STR-04 — Decision-critical content is present without client-side interaction · weight 3
Check: fetch the page without JavaScript, or read the initial HTML payload. Confirm pricing, requirements, eligibility, and next steps are present rather than behind a tab, an accordion, or a hydration step.
Why it matters: many agents read a fetched document rather than driving a browser. Content that only exists after an interaction does not exist for them.
STR-05 — Key actions have stable identifiers · weight 2
Check: confirm primary controls carry stable ids, data attributes, or ARIA labels that survive a design change and a CSS rebuild.
Why it matters: agents and their operators cache selectors. Hash-generated class names invalidate every cached path on each deploy.
STR-06 — Structured data is present and matches visible content · weight 2
Check: validate JSON-LD against schema.org and confirm each asserted value appears in the rendered page — dates, prices, and availability especially.
Why it matters: structured data that contradicts the page is worse than none, because it hands a machine a confident wrong answer. Google’s guidance is explicit that structured data should match the visible content (Google Search Central, AI features and your website).
STR-07 — Application state is legible in the document · weight 2
Check: after a state-changing interaction, confirm the new state appears in text or in ARIA state attributes, not only in colour, position, or animation.
Why it matters: an agent verifies its own work by re-reading the page. State expressed only visually leaves it unable to confirm whether its step succeeded — and an unverifiable step is one it will often repeat.
Dimension 3 — Machine-readable content (weight 17)
Is the substance of the site available in a form a machine can consume cheaply and exactly?
MRC-01 — Key pages have a clean text or Markdown representation · weight 4
Check: request the .md twin of a content URL, or request the canonical URL with Accept: text/markdown. Confirm the returned document carries the same substance as the HTML page rather than a stub.
Why it matters: a Markdown twin strips navigation chrome, cuts token cost, and removes the extraction step where meaning is most often lost.
MRC-02 — The machine-readable representation is advertised, not hidden · weight 3
Check: confirm content negotiation on the canonical URL, or a Link: <...>; rel="alternate"; type="text/markdown" header, or an in-page <link rel="alternate">.
Why it matters: a twin nobody can discover is not a discovery surface. Vercel’s engineering write-up makes the case for negotiation over separate URLs specifically because “content negotiation requires no site-specific knowledge” (Vercel, “Making agent-friendly pages with content negotiation”, 3 February 2026).
MRC-03 — Programmatic surfaces publish a machine-checkable contract · weight 3
Check: confirm a current OpenAPI document or JSON Schema for each public API, served at a discoverable URL, with a version matching the deployed service.
Why it matters: a schema is the only part of an API description a runtime can validate against. Prose describing an endpoint cannot catch a malformed call before it is sent.
MRC-04 — Publication and modification dates are visible and in metadata · weight 2
Check: confirm a visible date on the page and a matching datePublished / dateModified in structured data or HTTP headers.
Why it matters: answer engines weigh freshness. A page with no date forces a consumer either to discard it or to trust it blindly.
MRC-05 — Pages open with a self-contained answer · weight 2
Check: read the first 60 words under the h1 and under each h2. Confirm each is extractable as a standalone answer without the paragraph above it.
Why it matters: retrieval systems lift passages, not pages. A passage that depends on its predecessor becomes wrong the moment it is quoted alone.
MRC-06 — No decision-critical content is locked in images or unstructured PDFs · weight 3
Check: inventory diagrams, pricing tables, and specification sheets. Confirm each has a text equivalent in the page or in a linked structured document.
Why it matters: image-only content requires a vision pass, costs more, and is read less reliably than the same content in text.
Dimension 4 — Action safety (weight 18)
When an agent acts, are the consequences declared and the guardrails enforced by the system rather than by the prompt?
ACT-01 — Irreversible and high-consequence actions are labelled as such · weight 4
Check: for each write operation, confirm the tool description, API reference, or UI copy states whether the action is reversible, who can see it, and what it costs.
Why it matters: an agent cannot infer irreversibility. If the consequence is not stated, it gets discovered by being caused.
ACT-02 — Approval for consequential actions is enforced server-side · weight 4
Check: attempt the consequential action with a valid credential and no approval token. Confirm the server refuses, rather than relying on a client-side prompt or an instruction inside a tool description.
Why it matters: safety hints in tool metadata are advisory. The Model Context Protocol specification treats tool annotations as untrusted unless the server itself is trusted (MCP specification, 2026-07-28). Enforcement has to live in the service.
ACT-03 — Write operations are safely retryable · weight 3
Check: send the same write twice with the same idempotency key and confirm one effect. Confirm the mechanism is documented.
Why it matters: agents retry on timeout by default. Without idempotency, a network blip becomes a duplicate order.
ACT-04 — Agent identity is distinguishable from human identity · weight 3
Check: confirm the system can tell that a request came from an agent acting for a person — through delegated OAuth scopes, a distinct client identity, or a declared agent header — and that this is recorded.
Why it matters: attribution is the precondition for every other control. Rate limiting, audit, revocation, and honest analytics all depend on knowing who acted.
ACT-05 — Credentials are scoped to the task · weight 2
Check: confirm agent-facing access can be issued more narrowly than a full user session, and revoked independently.
Why it matters: a long-lived full-privilege key handed to an agent makes every downstream mistake maximally expensive.
ACT-06 — Rate limits and quotas are documented and expressed in responses · weight 2
Check: confirm published limits, machine-readable limit headers, and Retry-After on 429 responses.
Why it matters: an agent that cannot read its remaining budget will either back off far too much or keep going until it is blocked.
Dimension 5 — Recovery (weight 15)
When something goes wrong, can the agent work out what to do next — and can a person see what happened?
REC-01 — Errors are typed and named, not only prose · weight 4
Check: trigger the common failure modes and confirm each returns a stable machine-readable code alongside the human message.
Why it matters: a stable code is what lets an agent branch. A sentence whose wording changes between releases cannot be matched on.
REC-02 — Errors name the next viable step · weight 3
Check: read the error payloads. Confirm each says what would make the request succeed, or which alternative route exists.
Why it matters: most agent loops fail not because a call errored, but because the error gave no basis for a different second attempt.
REC-03 — Validation failures identify the specific field and constraint · weight 2
Check: submit a partially invalid payload. Confirm the response names the field, the rule it violated, and the accepted range or format.
Why it matters: a generic “invalid request” turns a one-shot correction into a guessing loop across every field.
REC-04 — Partial progress is resumable · weight 3
Check: interrupt a multi-step flow. Confirm the state can be queried and continued rather than restarted from the beginning.
Why it matters: long agent tasks are interrupted routinely. A non-resumable flow converts every interruption into repeated side effects.
REC-05 — Agent actions are inspectable by the person responsible · weight 3
Check: confirm a person can see what an agent did on their behalf — timestamps, inputs, outcomes — and can reverse or escalate from that view.
Why it matters: delegation without an audit view moves work out of the person’s sight. Recovery is a product surface, not only a log file.
Dimension 6 — Policy signals (weight 12)
Has the organisation said, in machine-readable terms, what AI systems may do with this content and these capabilities?
POL-01 — A usage preference for AI systems is stated in machine-readable form · weight 3
Check: confirm a declaration in robots.txt, in HTTP headers, or in a well-known document — not only in prose terms of service.
Why it matters: a policy only a lawyer can read is not a policy a crawler can honour.
POL-02 — Distinct uses are distinguished, not collapsed · weight 3
Check: confirm the declaration separates indexing for search, use as input to a generated answer, and use as training data — rather than issuing one blanket allow or deny. Cloudflare’s Content Signals Policy is the most widely deployed vocabulary for exactly this split (Cloudflare, Content Signals Policy).
Why it matters: most organisations want different answers for these three uses. A single directive forces one answer for all of them.
POL-03 — Terms address agents acting on a user’s behalf · weight 2
Check: read the terms of service for language covering automated access performed for a logged-in user, as distinct from scraping.
Why it matters: terms written against scrapers routinely prohibit, by accident, the assistant the customer is deliberately using.
POL-04 — A contact route exists for agent and crawler operators · weight 2
Check: confirm a published contact address or endpoint for access problems, referenced from robots.txt or the capability manifest.
Why it matters: without a route, an operator whose agent is blocked has no option other than to work around the block.
POL-05 — Policy signals agree across surfaces · weight 2
Check: compare robots.txt, the machine-readable preference declaration, the capability manifest, and the terms of service for contradictions.
Why it matters: contradictory signals get resolved by whoever is reading them — which means the organisation is no longer choosing its own policy.
How does this relate to the readiness tools that already exist?
Several organisations published agent-readiness models between April and June 2026. They are not competing measurements of the same thing; they measure different objects, which is why their scores cannot be compared. We list them here without ranking them.
| Framework | Publisher | Object being scored | Shape |
|---|---|---|---|
| Agent readiness score | Cloudflare | A public website, from edge-observable signals | Four dimensions — Discoverability, Content, Bot Access Control, Capabilities; public checker at isitagentready.com reporting a level 0–5 across 22 checks |
| AXIS | Netlify | A live agent run against a scenario | Four dimensions: goal achievement, environment, service, agent behaviour; 0–100 with an inspectable transcript |
| Agent Readiness Model | Factory | A code repository | Nine technical pillars, five maturity levels (L1 Functional → L5 Autonomous) |
| Agentic Readiness | AWS | An enterprise application, as tool and as resource | Five pillars: API and interface readiness, security and identity, data handling, operational resilience, observability |
| Lighthouse agentic-browsing audits | Google Chrome | A single page, in an existing audit runner | Experimental category covering WebMCP integration, an llms.txt discoverability check, accessibility for agents, and layout stability; explicitly not a weighted 0–100 score |
| This rubric | agentexperience.tech | A public site or product’s declared surfaces | 36 weighted criteria, six dimensions, hand-runnable |
Two structural distinctions are worth holding onto. First, behavioural versus structural: AXIS scores an observed run, this rubric scores declared surfaces. A behavioural score tells you what happened once; a structural score tells you what any agent can rely on. You want both, and neither substitutes for the other. Second, repository versus public web: Factory’s model and AWS’s model score things inside an organisation. Cloudflare’s, Lighthouse’s, and this one score what the outside world can see.
Where this rubric differs most from the edge-observable checkers is Dimensions 4 and 5. Action safety and recovery cannot be measured from outside without a credential and a willingness to break something, so automated public scanners largely skip them. They are also, in our experience of reviewing agent workflows, where most real failures live.
What does the current baseline look like?
Honest answer: thinner than the discourse suggests, and the public evidence base is small.
The one large public measurement we have found is Cloudflare’s, published on 17 April 2026 alongside its agent readiness score and the isitagentready.com checker (Cloudflare, “Introducing the Agent Readiness score”, André Jesus and Vance Morrison). Read the population carefully: Cloudflare took the 200,000 most visited domains on Cloudflare Radar and then filtered out categories where agent readiness is not relevant — redirects, ad servers, tunnelling services. The post does not state the post-filter count, so “200,000 domains scanned” is a slight overstatement of what was published.
Against that population, Cloudflare reported:
| Signal | Finding, as published 17 April 2026 |
|---|---|
robots.txt |
“nearly universal — 78% of sites have one — but the vast majority are written for traditional search engine crawlers, not AI agents” |
| Content Signals | “4% of sites have declared their AI usage preferences in robots.txt” |
| Markdown content negotiation | “passes on 3.9% of sites” |
| MCP Server Cards and API Catalogs (RFC 9727) | “together appear on fewer than 15 sites in the entire dataset” |
Note that the last figure is a combined count for the two surfaces, not fewer than 15 of each.
Two things follow from a baseline this low. Adopting even the Discovery dimension puts a site ahead of most of the web on those specific signals — which is a statement about scarcity, not about outcomes. And a rubric built only on the signals that an edge scanner can see would be measuring the easy half of the problem.
One correction worth carrying, since it circulates widely: isitagentready.com describes itself, in its own machine-readable server description, as reporting an “agent readiness level (0-5) across 22 checks”. Several third-party write-ups describe it as producing a 0–100 score with named tiers. We found no support for that in either the Cloudflare post or the live tool.
How to run the rubric in an afternoon
- Fetch the four files.
robots.txt,sitemap.xml,llms.txt, and anything under/.well-known/. Record the status code and the content type for each. That settles most of Dimension 1. - Pick one real task. Not the homepage — the task a person would actually delegate. Walk it as an agent would: fetch the page without JavaScript, read the accessibility tree, list the controls and their names. That settles Dimension 2.
- Ask for the machine-readable copy.
curl -H "Accept: text/markdown"on three content URLs, and look for the schema behind any API. Dimension 3. - Try to break something safely. In a staging environment, call a write endpoint twice, call it without approval, and submit an invalid payload. Dimensions 4 and 5 — this is the part most teams skip, and the part that finds the most.
- Read your own policy files against each other. Dimension 6 usually takes ten minutes and usually turns up a contradiction.
- Publish the six subscores, the date, and the method. Then re-run it when a discovery surface, an API contract, or an approval flow changes.
Changelog
- v0.1 — 2026-09-01. First public draft. 36 criteria, six dimensions, weights totalling 100. Open for comment. Weights are a judgement call and are the part most likely to change in v0.2.
Frequently asked questions
What is agent readiness? Agent readiness is the degree to which a website or product makes its content, capabilities, consequences, and policies legible to a software agent working on a person’s behalf. It is assessed by inspecting declared surfaces — files, headers, markup, schemas, error payloads — rather than by asking how the product feels to use.
How is this rubric different from a vendor’s agent-readiness score?
This rubric is structural and open: it scores what a site declares, publishes its weights, and can be run by hand with a browser and curl. Vendor tools have different scopes — a code repository, a live agent transcript, an enterprise application, or edge-observable signals. They answer different questions and their scores are not comparable with each other or with this one.
Does a high score mean agents will succeed on my site? No. The rubric measures declared surfaces, not outcomes. A site can score well and still confuse an agent through unclear copy; a site can score poorly and still be usable by a capable model. Treat the score as an inventory of what you have made explicit, and measure outcomes separately with real task runs.
How often should we re-run it? Whenever a discovery surface, an API contract, or an approval flow changes, and on a fixed cadence otherwise. Several criteria fail through drift rather than absence — an index pointing at deleted URLs, or a manifest describing an endpoint that has since moved.
A readiness score is a description of what you have declared, not a promise about what agents will do with it.