AXC-D019: Example does not match the schema

A published example call fails the schema it is meant to illustrate. An agent can use an example as the template for its own call, so a broken example leads to a broken call. The agent then gets an error for doing exactly what the documentation showed.

What it checks

ax-check validates each published example against the input schema it belongs to:

• MCP: examples in an examples or inputExamples array on the tool object, when a server publishes one.
• OpenAPI: the example and the examples (each value) of the JSON media type in an operation’s request body.

Up to five examples per tool or operation are checked. This mirrors the catalog.invalid_example check in the audit_agent_path tool on agentexperience.tech, which uses the same shallow schema subset.

Why it matters

An example is the most concrete thing an agent sees about a tool. When the example says "currency": "euro" and the schema only allows "EUR", "USD" and "GBP", the agent has two sources that disagree, and the copyable one is wrong. A mismatch usually means the schema changed and the example did not, which is a form of drift.

Machine-checkable constraints help agents only when the published material agrees with them. In the SilentProbe paper (Li, Ye, Guo and Dang, arXiv 2609.00035, 29 August 2026), constraints declared in the schema produced an honest error in 111 of 111 cases (the paper’s claim). The paper did not test published examples. Applying its result to them is design rationale: an example that contradicts the schema invites a call that the schema rejects.

How to fix

• Run every published example through your schema validator in your test suite.
• Fix the example, or fix the schema if the example shows the intended behaviour.
• Remove examples that you cannot keep up to date.

Example

Before

{
  "name": "create_invoice",
  "inputSchema": {
    "type": "object",
    "properties": {
      "customer_id": { "type": "string", "description": "The customer's ID." },
      "amount": { "type": "number", "minimum": 0, "description": "Total in the invoice currency." },
      "currency": { "type": "string", "enum": ["EUR", "USD", "GBP"], "description": "ISO 4217 currency code." }
    },
    "required": ["customer_id", "amount", "currency"],
    "additionalProperties": false
  },
  "inputExamples": [
    { "customer": "cus_123", "amount": "120.00", "currency": "euro" }
  ]
}

After

{
  "inputExamples": [
    { "customer_id": "cus_123", "amount": 120, "currency": "EUR" }
  ]
}

The “After” shows only the corrected example; the schema is unchanged.

How ax-check detects it

ax-check uses a shallow subset of JSON Schema, the same subset as the agentexperience.tech audit. It checks that:

• the example is an object;
• every property in required is present;
• when additionalProperties is false, the example has no properties outside properties;
• each top-level value matches its property’s type, enum and const;
• strings respect minLength and maxLength, numbers respect minimum and maximum, and arrays respect minItems, maxItems and the type or enum of their items.

The finding lists up to three errors per example.

If the schema is not a plain object schema, or uses $ref, oneOf, anyOf, allOf, not, if, then, else, patternProperties or a list of types (at the top level or directly on a property), ax-check skips that tool’s examples rather than guess. A top-level anyOf whose branches only list required is still checked.

Like AXC-D018 and the audit, ax-check does not treat an open schema as a contradiction. An example property that is not defined in properties is reported only when the schema is closed.

Known false negatives: nested objects are not validated, and format, pattern, exclusiveMinimum, exclusiveMaximum, multipleOf and uniqueItems are not checked. OpenAPI examples on path, query or header parameters, and the examples keyword inside a schema, are not read. Use a full JSON Schema validator in your own tests for complete coverage.

If the finding does not apply, silence it with --disable AXC-D019.

Sources

• Paper: Li, Ye, Guo and Dang, “SilentProbe: Measuring Silent Failure in Production APIs Used as Agent Tools”, arXiv 2609.00035, 29 August 2026. https://arxiv.org/abs/2609.00035. The paper’s claim: machine-checkable constraints gave an honest error in 111 of 111 cases, while prose-only constraints failed silently in 44 of 61. Endpoints were reached through one aggregation layer with which the authors declare an affiliation.
• Specification: OpenAPI Specification v3.1.0, OpenAPI Initiative, 15 February 2021. https://spec.openapis.org/oas/v3.1.0. Defines example and examples on the Media Type Object.
• Guide: Agent-readable documentation for AI agents, agentexperience.tech, retrieved 2026-10-08. https://agentexperience.tech/insights/make-documentation-legible/. Documentation for agents has to be kept true as the system changes, including its examples.
• Related check: the agentexperience.tech audit_agent_path tool reports the same defect as catalog.invalid_example.

Related evidence

Records in the AX evidence register that share a pattern tag with this rule. A shared tag means the record is about the same pattern, not that it tests this rule. Read the evidence class before the number.

• EV-0001: An enum in the schema ends silent failures from example-only vocabularies (Preprint). SilentProbe (preprint) reports that a vocabulary a parameter description only exemplified ("e.g.") was missed on 88 of 88 attempts across twelve models, and that promoting it into the schema cut the failure to 0 of 89.
• EV-0002: Constraints stated only in prose produce silent failures on live APIs (Preprint). SilentProbe (preprint) reports that only 7.5% of 2,501 public OpenAPI documents declare an enum, and that on live endpoints machine-checkable constraints returned an honest error in 111 of 111 cases while prose-only constraints failed silently in 44 of 61.
• EV-0011: Coding agents mostly read instruction files, not docs sites (Preprint). An observational study of 557 agentic coding sessions (preprint) found that instruction files and working notes made up 60.5% of agents' documentation interactions, against 10.6% for classical technical documentation and 1.3% for API references.
• EV-0012: Compact documentation did not help when the source was present (Preprint). Across two model families and ten repositories, with a positive control, a preprint found that neither compact natural-language documentation nor retrieved context helped coding agents resolve issues better than the issue alone when the source code was present.
• EV-0027: A capable agent skipped the index and guessed the page (Preprint). A preregistered ablation on a 709-page Markdown wiki (preprint) found that a capable tool-using agent never loaded the compact catalogue index, inferring page paths from the question instead, while retrieval-based access kept answer quality non-inferior and cut cost by about a third to over half.
• EV-0029: A warning that names the failing plan redirected agents; a tip did not (Vendor measurement). Microsoft reports that a documentation tip pointing to the right tool got 1 of 5 agent runs to use it, while a warning naming the agent's failing approach ('Manually updating package.json alone will result in build failures') got 5 of 5.

Related guides

• Write tool descriptions an agent can act on
• Write agent-readable documentation