Evidence record EV-0020

Praise and list order move tool selection

A preregistered preprint with two small OpenAI models found that stacked praise in a tool description raised its pick rate by about 43 percentage points, and that with identical listings the first-listed tool was picked about 72 points more often.

Preprint · Measurement · Retrieved

Evidence class: Preprint. Unreviewed. Many are by one author or a small team, and some authors have a stake in the result. Most AX research is in this class today. Pattern tags: description, selection.

Effect, as the source reports it

  • Pick rate with stacked praise (four kinds combined). Baseline: Listing without praise. With the change: Listing with stacked praise. Direction: increase. Size: About +43 percentage points, matching or beating a verifiable specification. Sample: Two OpenAI models; preregistered study.
  • Pick rate of the first-listed tool with identical listings. Baseline: Listed second. With the change: Listed first. Direction: increase. Size: About +72 percentage points. Sample: As above.
  • Picking the capable tool on numeric-limit tasks. Baseline: Structured limit fields. With the change: Structured fields plus provider sales text. Direction: decrease. Size: Sales text reduced or erased the gain from structured fields. Sample: As above.

Agent profile

  • Note on models: Two small OpenAI models. The abstract does not name them.

Conflicts of interest

None declared in the abstract. The full text was not checked for a competing-interest statement.

Source

Agent-Facing Information Design in LLM Tool Registries: A Preregistered Test of Rhetoric, Position and Structure, arXiv, Haochuan Kevin Wang, Zechen Zhang, 12 April 2026 (v2, revised 2026-09-30 (adds the preregistered study)), arXiv:2605.23916. Retrieved ; verification: abstract-only.

Every number in this record was checked against the live arXiv abstract page on 2026-10-08. The full text was not re-checked.

Limitations

  • Preprint, not peer reviewed.
  • Two small models.
  • The authors call the results provisional until blind phrase ratings are complete; only stacked praise, not any single kind, replicated on held-out domains.

For designers

Tool descriptions are a ranking surface. Write limits as structured fields and leave out sales language; if you run a registry, randomise the order and hide promotional text from agents.

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0020" and this page's address, https://agentexperience.tech/evidence/ev-0020/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.