# Find evidence for what to improve next

> For teams that need to check how ready one job is, measure a change, and decide what to fix first.

Section: Evaluation and improvement  
Canonical: https://agentexperience.tech/insights/topics/evaluation/

For teams that need to check how ready one job is, measure a change, and decide what to fix first.

You only know whether agent experience got better by checking it on real tasks. This group covers the Open Agent-Readiness Rubric, scoring the decisions inside a workflow rather than only its answer, measuring a change against a bare baseline, and running a review of one workflow from start to finish. It also compares the readiness scores other people publish and collects public examples you can check for yourself. Use these when you have to decide what to fix first and then show whether the fix worked.

[Read the first guide](https://agentexperience.tech/insights/agent-readiness-rubric/)

## The guides, in order

- [The Open Agent-Readiness Rubric](https://agentexperience.tech/insights/agent-readiness-rubric/): Check one real job across six tracks, from finding the page to handing the result back, and get a profile instead of a score. 25 min read
- [Evaluate the decisions in an agent workflow](https://agentexperience.tech/insights/evaluating-agent-workflows/): Evaluate tool choice, handoffs, and recovery alongside the final result. 8 min read
- [How do you measure agent experience?](https://agentexperience.tech/insights/measure-agent-experience/): The key agent experience metrics in four families, how to compare a change against a bare baseline, and the claims none of them can support. 17 min read
- [Run an Agent Experience review](https://agentexperience.tech/insights/run-an-ax-review/): A repeatable way to inspect one meaningful workflow and choose the next product improvement. 9 min read
- [Agent evaluation loop: run, judge, triage, fix, repeat](https://agentexperience.tech/insights/build-a-reliable-agent-evaluation-loop/): Six stages, a fixed task set, and a failure taxonomy that names the first decision that broke instead of the last error. 11 min read
- [Agent readiness scores compared](https://agentexperience.tech/insights/agent-readiness-scores-compared/): Cloudflare, Lighthouse, Is Agentic, AFDocs scores, AXIS and the 2027.dev tier list: what each one scores, how, and what it cannot tell you. 16 min read
- [Agent experience examples you can check](https://agentexperience.tech/insights/agent-experience-examples/): Seventeen annotated, public AX design decisions, mapped to the rubric, with a checklist and a template for recording your own. 16 min read

## Rules on these topics

[All rules](https://agentexperience.tech/checks/)

- [Description length outlier](https://agentexperience.tech/checks/axc-d016/): The description is far longer than the others in the same catalogue, which costs context on every turn.
- [Context footprint estimate](https://agentexperience.tech/checks/axc-d017/): Reports the bytes and approximate tokens the whole surface costs an agent before it does any work.

## Evidence on these topics

[All evidence](https://agentexperience.tech/evidence/)

- [Agent scaffolding, not MCP versus CLI, drove cost](https://agentexperience.tech/evidence/ev-0010/): A controlled comparison (preprint) found that on one git task the agent scaffolding drove cost more than MCP versus CLI: CLI-only scaffoldings were 5.0x to 28x cheaper, and paired MCP-to-CLI cost ratios ranged from 0.43x to 29x.
- [A model with cheaper tokens cost 3.7x more per run](https://agentexperience.tech/evidence/ev-0031/): Microsoft reports that on SharePoint Framework upgrade tasks Claude Sonnet 5 cost $2.01 per run against $0.55 for Claude Sonnet 4.6, 3.7x more despite 33% lower per-token prices, while on architecture tasks it was 12% cheaper.
- [Run-to-run variance dominates, and a verification tool beats a verification prompt](https://agentexperience.tech/evidence/ev-0025/): A preprint reports that about 54% of outcome variance in its coding-agent benchmark came from repeating the same configuration, and that a dedicated verification tool changed verification behaviour substantially where a prompt asking the agent to verify had little effect.
- [Declared completion and logged completion differ by up to 41 points](https://agentexperience.tech/evidence/ev-0008/): WebPageBench (preprint) reports a gap of up to 41 points between the tasks web agents declared finished and the tasks the site's own event log confirmed, with one configuration declaring every task finished while meeting the conditions on 59%.
- [Harnesses alter shell calls and the wrong action runs silently](https://agentexperience.tech/evidence/ev-0024/): A preprint reports that in 47,828 production shell calls, Claude Code's Bash tool changed 12.0% of calls carrying code, escape sequences or long text, and that for 80.7% of calls whose backslashes were changed the wrong action ran with no reported error.
- [A JSON input mode for a CLI raised agent cost 4x to 11x](https://agentexperience.tech/evidence/ev-0028/): Microsoft reports that on a synthetic CLI, run five times per model in GitHub Copilot Chat, every model was correct 5/5 with individual arguments, while a --json payload mode cost 4x to 11x more per task and dropped Claude Haiku 4.5 to 2/5 and MAI-Code-1-Flash to 3/5.

[Handle approvals, failures, and handoffs](https://agentexperience.tech/insights/topics/workflows/)

[All guides](https://agentexperience.tech/insights/)