On this page · 8 sections
Agent-facing work is a chain of decisions. The outcome matters, and so do the steps that lead to it: finding a capability, selecting a tool, forming the right input, knowing when to ask for help, and recognising when to stop.
Why is final-answer accuracy not enough?
Because the same score can cover very different products. Consider two runs of the same task. In the first, the agent reads the right documentation, calls the right tool once with well-formed arguments, and returns. In the second, it calls a neighbouring tool, gets a partial result, retries twice, works around a permission it should have escalated, and arrives at the same output. An outcome-only check records two successes. Only one of them will survive a slow dependency, a stricter policy, or a slightly different phrasing of the request.
The evaluation literature has been moving away from outcome-only scoring for this reason. AgentBoard (2024) reports progress rate and grounding accuracy alongside success rate. Mind2Web 2 (2025) uses an agent-as-a-judge rubric to score long-horizon search where the final answer is dynamic. API-Bank (Li et al., EMNLP 2023) scores API planning, retrieval, and calling as separate tasks over 73 runnable APIs, on the explicit basis that tool use is not one skill and its parts fail independently. Several web-agent studies collect step-level human feedback specifically to surface failure modes that success rates hide.
You do not need a benchmark for your own review, but the decomposition those benchmarks use is the right shape for it.
Watch the path
Start by watching a few real runs of the workflow. Look for repeated detours, uncertain handoffs, avoidable retries, and changes in behaviour when context is incomplete. Those patterns often tell a team more than a single success rate.
Five decision points are worth naming, because each has a different fix.
Discovery. Did the agent find the capability at all? If the agent cannot find a capability, it might as well not exist. The fix is usually a page, an index, or a catalogue entry.
Selection. Did it choose the right capability among neighbours? Near-miss selection is easy to misdiagnose, because it looks like reasoning and is usually wording. The fix lives in Write tool descriptions an agent can act on.
Input formation. Were the arguments well-formed and complete? Malformed parameters and missing required context are recurring failure modes across the tool-use literature. The fix is usually a schema with real descriptions and constraints, or a preflight check.
Boundary and stopping. Did the agent respect the permission model, and did it stop where it should? Silent overreach and unnecessary escalation are both failures, and only the second shows up in a support queue.
Recovery. When something failed, did the next attempt have more information than the last? A retry that repeats the same call with the same inputs is only repetition. See Record retries as product history.
Record these as plain sentences, not scores, on the first pass. “The agent could not tell export_report from share_report” is more actionable than “selection accuracy: 0.6”.
Test the moments that carry risk
Not every step needs the same scrutiny. Focus on choices that create side effects, expose information, change a customer outcome, or become expensive to correct, because that is where product boundaries matter most.
A simple way to allocate attention: for each step, ask whether an incorrect action is reversible, visible, and contained.
- Reversible, visible, contained: a read, a draft, a search. Let it run and sample the results.
- Irreversible or externally visible: a payment, an email to a customer, a deletion, a publish. Every one of these needs a designed approval, and a confirmation dialog is not enough. See Human approval workflows for AI agents.
- Uncontained: anything the agent can do after ingesting untrusted content. Prompt-injection research is consistent that retrieved text can steer behaviour into unintended calls (Greshake et al., 2023), and that the structural mitigation is narrowing the consequential actions available after untrusted input (Beurer-Kellner et al., 2025). Evaluate this deliberately: give the workflow a hostile document and see what it is still able to do.
The current MCP specification (2026-07-28) makes the same separation: tool annotations such as read-only or destructive hints are untrusted unless the server is trusted, and the harness must still validate and authorise. So when you evaluate a boundary, test how it is enforced instead of trusting its label.
Measure reliability, not one good run
One successful run tells you the workflow is possible, but not that it is reliable.
Three techniques from public evaluation practice transfer well to product work:
Repeat the same task. The τ-bench work (Yao et al., ICLR 2025) introduced pass^k (a task counts as successful only if the agent succeeds on all k attempts) as a way to describe reliability in multi-turn, policy-constrained settings rather than best-run performance. Running each of your key tasks a handful of times and looking at the worst run is a cheap approximation of the same idea, not a reproduction of the metric.
Perturb the inputs. Vary the phrasing, omit a field, and change the order of information. ReliabilityBench (2026) formalises this as a reliability surface over perturbations and injected tool failures rather than a single scalar. You do not need the formalism, just three variants of each task.
Inject faults. Simulate a timeout, a rate limit, a partial response, and a schema change. This is the only way to find out whether your recovery path actually exists.
Record cost, step count, and latency alongside correctness. A workflow that succeeds reliably in forty steps is a different product from one that succeeds reliably in four, and only one of them will survive a pricing review.
Keep deterministic and judgement checks separate
Some conditions are facts. Did the run use only allowed capabilities? Did the output match the required schema? Did it stay inside the permitted scope? Check these deterministically, in code, every time.
Other questions need judgement. Was the result appropriate? Was the path understandable? Did the handoff happen at the right moment? These need a rubric and a reviewer, either a person or a model acting as judge with its limits acknowledged.
Do not blend them. A judge that also decides schema compliance becomes an unreliable linter, and a linter that also decides appropriateness becomes a false comfort. When a model is used as a judge, it becomes part of the measurement and can drift. Public benchmarks that use agent-as-a-judge rubrics list judge quality as a validity threat.
Do not compare yourself to a leaderboard
It is tempting to anchor on a published number, and it is a mistake. A public agent score is a systems result. It reflects a model, a harness, an environment version, a verifier, a step budget, and a retry policy, and 2026 work on harness sensitivity exists specifically because the scaffold can shape the score. Benchmark releases are frequently not comparable to their own earlier versions: the τ-bench repository notes that results before a given release are not comparable with later graded results in affected domains, and OSWorld 2.0 (2026) is explicitly not directly comparable to OSWorld 1.0 unless the release matrix matches.
The useful move is to borrow the method (decomposition, repeats, perturbation, fault injection, execution-based checks) and apply it to your own tasks, with your own versions recorded. What you can defensibly say is “this workflow, at these versions, behaves this way.” That is more useful to your team than a rank.
Use the result to improve the system
Evaluation should lead to a better next decision: simplify the tool set, clarify the context, adjust a handoff, or define the condition that should pause the workflow.
Every finding should land in one of four places:
- A wording change: a description, a schema field, a heading, a label.
- A structural change: a split capability, a new page, a preflight check.
- A boundary change: a narrower scope, an added approval, an encoded stopping condition.
- A standing check: the failure becomes a permanent case so it cannot quietly return.
The last one turns a one-off evaluation into a loop. An agent evaluation loop turns one failure into a standing check covers the mechanics; Run an Agent Experience review covers how to find the cases in the first place.
Frequently asked questions
Why is final-answer accuracy not enough for agent evaluation? Because a correct answer can come from a bad path. An agent can pick a near-miss tool, retry three times, skip a permission boundary, and still land on the right result, and an outcome-only check scores that identically to a clean run. Public benchmarks have moved the same way: AgentBoard reports progress rate and grounding accuracy alongside success rate, precisely because pass-or-fail hides how the result was reached.
What should we measure in an agent workflow? Measure the decisions that carry consequence: whether the right capability was selected, whether inputs were well formed, whether boundaries were respected, whether the agent stopped when it should have, and whether recovery was possible. Then measure reliability across repeats rather than a single run, and record cost, steps, and latency alongside correctness.
What is pass^k and why does it matter?
pass^k scores a task as successful only if the agent succeeds on all k attempts, rather than any one of them. It was introduced in the τ-bench work (Yao et al., ICLR 2025) as a way to describe reliability in multi-turn, policy-constrained settings. It matters because production users experience the distribution of runs, not the best run.
Can we compare our agent’s score to a public benchmark number? Not directly. A published agent score reflects a model plus a harness plus an environment version plus a verifier, and benchmark releases are often not comparable across their own versions. Use public benchmarks to borrow evaluation design (decomposition, perturbation, repeats) rather than to position your product against a leaderboard.
A useful evaluation leaves the team knowing what to change next.