Evidence record EV-0008

Declared completion and logged completion differ by up to 41 points

WebPageBench (preprint) reports a gap of up to 41 points between the tasks web agents declared finished and the tasks the site's own event log confirmed, with one configuration declaring every task finished while meeting the conditions on 59%.

Preprint · Measurement · Retrieved

Evidence class: Preprint. Unreviewed. Many are by one author or a small team, and some authors have a stake in the result. Most AX research is in this class today. Pattern tags: false-success, evaluation, measurement.

Effect, as the source reports it

  • Gap between declared and log-confirmed task completion. Baseline: Declared finished. With the change: Confirmed by the event log. Direction: decrease. Size: Up to 41 points; one configuration declared 100% and met the conditions on 59%. Sample: 152 tasks; public leaderboard of 24 model-harness pairs.

Agent profile

  • Note on models: Six browser/DOM harness configurations and five screenshot-only GUI-agent families. The abstract does not name the models.

Conflicts of interest

None declared in the abstract. The full text was not checked for a competing-interest statement.

Source

WebPageBench: Event-Level Verification and Controlled UI-Variant Generation for Web Agents, arXiv, Anton Emelyanov, Maria Tikhonova, Zaven Martirosian, Sergei Averkiev, Alena Fenogenova, 28 September 2026, arXiv:2609.35026. Retrieved ; verification: abstract-only.

Every number in this record was checked against the live arXiv abstract page on 2026-10-08. The full text was not re-checked.

Limitations

  • Preprint, not peer reviewed.
  • Six instrumented mock sites with brand identifiers removed.

For designers

Measure success from your own system's events, not from what the agent says. Emit an event for each step that completes so it can be checked.

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0008" and this page's address, https://agentexperience.tech/evidence/ev-0008/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.