Evidence record EV-0027

A capable agent skipped the index and guessed the page

A preregistered ablation on a 709-page Markdown wiki (preprint) found that a capable tool-using agent never loaded the compact catalogue index, inferring page paths from the question instead, while retrieval-based access kept answer quality non-inferior and cut cost by about a third to over half.

Preprint · Measurement · Retrieved

Evidence class: Preprint. Unreviewed. Many are by one author or a small team, and some authors have a stake in the result. Most AX research is in this class today. Pattern tags: docs-for-agents, discovery, context-budget.

Effect, as the source reports it

  • Whether the agent loaded the catalogue index. With the change: Never loaded in the pilot; the agent inferred page paths and read pages directly. Direction: not-applicable. Size: Never. Sample: Pilot.
  • Cost of answering, retrieval arm versus index baseline. Baseline: Index baseline. With the change: Retrieval arm. Direction: decrease. Size: About a third (self-routing agent) to well over half (catalogue preload); all confidence intervals exclude zero. Sample: Four corpus arms crossed with three access conditions; blind grading against verified gold answers.
  • Answer quality. Baseline: Index baseline. With the change: Retrieval arm. Direction: no-change. Size: Non-inferior within the preregistered margin. Sample: As above.

Agent profile

  • Note on models: A protocol-constrained agent, a free self-routing agent and a catalogue-preload regime. The abstract does not name the models.

Conflicts of interest

None declared in the abstract. The full text was not checked for a competing-interest statement.

Source

Progressive Disclosure for LLM-Maintained Wiki Knowledge Bases: a Preregistered Ablation, arXiv, Theodore O. Cochran, 6 July 2026, arXiv:2607.04576. Retrieved ; verification: abstract-only.

Every number in this record was checked against the live arXiv abstract page on 2026-10-08. The full text was not re-checked.

Limitations

  • Preprint, not peer reviewed.
  • One wiki maintained by an LLM.
  • Single author.
  • Does not test public web standards such as llms.txt.

For designers

Do not assume an agent will read an index or catalogue page before acting; a capable agent may guess the path. Stable, guessable addresses and targeted retrieval did more here than the index.

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0027" and this page's address, https://agentexperience.tech/evidence/ev-0027/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.