AX Workflow Review brief

1. Define the workflow

Start with a workflow a person would plausibly delegate and a success condition the team can observe. The scope is what keeps the review from becoming a general argument about model quality.

Workflow name

Name the task tightly enough that everyone knows which run is being reviewed.

Person delegating the work

Use a role or audience, not a real customer name.

User goal

State what the person wants done and why the result matters.

Agent starting context

List what the agent receives before it begins: prompt, files, account state, docs, or prior messages.

Success criteria

Define the observable end state that would make the run acceptable.

Stop / ask condition

Name when the agent should pause, ask, refuse, or return control instead of continuing.

Out of scope

Protect the review from expanding into every possible model, prompt, product, or workflow question.

2. Map the agent-facing surfaces

Include only the artifacts this workflow touches. For each surface, record the owner, the current risk, and the evidence that would settle whether it helped or hurt the run.

3. Collect the run record

A useful review needs the path, not just the outcome. Capture the sequence of decisions, interventions, retries, and terminal state so the team can see where the product surface affected the run.

• Task ID or scenario name.
• Model, agent, or harness version if it matters to the review.
• Date of run.
• Starting prompt or user request summary.
• Tool calls in order, with inputs redacted when needed.
• Human interventions and why they happened.
• Retries, including what changed between attempts.
• Terminal state: success, partial progress, refused, unsafe, or blocked.
• Safe evidence: screenshots, logs, docs links, diffs, or transcript excerpts that can be shared internally.

Keep credentials, customer data, private traces, and confidential operational detail out of any public copy.

4. Separate the first failure from downstream effects

Mark the first visible failure before listing what happened later. For each stage you inspect, keep four fields with the observation: observed failure, evidence, severity, and candidate owner.

1. DiscoveryCould the agent find the right path or capability?
2. RetrievalDid the agent retrieve the context it needed?
3. Tool choiceDid it choose the right tool or action?
4. Tool inputsWere required fields, constraints, and side effects clear?
5. ApprovalWas permission scoped, enforced, and auditable?
6. RecoveryDid the failure response support a better second attempt?
7. HandoffCould a person trust and continue from the returned work?

5. Choose the change and next test

The output is not a long report. It is one prioritized finding, one surface change, and one way to check whether the next attempt changed in the intended direction.

• Finding: the specific decision point or surface that created friction.
• Product change: the smallest edit that should alter the next attempt.
• Expected signal: what should be different in the next run if the change helped.
• Test: the replay, regression, or review step that will check that signal.
• Owner and status: who will make the change and when it will be reviewed again.

Next action

• One change to make now.
• One regression or replay case to keep.
• One open question to resolve before expanding scope.
• One date to re-run the workflow.

Use the brief as a worksheet, not a scorecard or prediction of rankings, citations, or universal agent success.