Evidence record EV-0030

Adding the context7 MCP server gave no lift; its tools went unused

Microsoft reports that adding the context7 MCP server to an anti-hallucination skill gave no meaningful lift on an SPFx upgrade: its tools did not load in 3 of 5 runs and were not called in the other 2, while telling the agent to use CLI for Microsoft 365 raised configuration correctness from 30/80 to 75/80.

Vendor measurement · Null result · Retrieved 8 October 2026

Evidence class: Vendor measurement. The vendor chose the tasks, ran the evaluation and wrote it up. Nobody else has reproduced it unless the record says so. Pattern tags: propensity, docs-for-agents, skills.

Effect, as the source reports it

Configuration correctness (points out of 80). Baseline: Bare baseline: 30/80. With the change: Anti-hallucination skill: 37/80; skill plus context7: 41/80; told to use CLI for Microsoft 365: 75/80. Direction: increase. Size: context7 added no meaningful lift over the skill; the CLI instruction reached 75/80. Sample: Five runs per configuration.
context7 tool use. With the change: Tools not loaded into context in 3 of 5 runs; available but never invoked in 2 of 5. Direction: not-applicable. Size: 0 of 5 runs used it. Sample: Five runs.
Dependency currency (points out of 50). Baseline: Bare baseline: 34/50. With the change: Skill: 38/50; skill plus context7: 39/50; CLI instruction: 49/50. Direction: increase. Size: 34/50 to 49/50 with the CLI instruction. Sample: Five runs per configuration.

Agent profile

Models: Claude Sonnet 4.6.
Harness: GitHub Copilot Chat in Visual Studio Code.
Extensions: SPFx anti-hallucination skill; context7 MCP server.
Operating system: Windows.

Conflicts of interest

Microsoft authors evaluating with Microsoft tools (GitHub Copilot Chat, VS Code) and, where noted, Microsoft products. Microsoft reports the results itself. The task is an upgrade of Microsoft's SharePoint Framework; context7 is a third-party product.

Source

Behind SPFx Dev Skills: testing what agents know and fixing what they miss, Microsoft 365 Developer Blog, 2 September 2026. Retrieved 8 October 2026; verification: verified-live.

The full primary source was fetched on 2026-10-08 and every number in this record was found in it. The post page names no individual author; published 2026-09-02 per its metadata.

Limitations

Vendor-run evaluation, not peer reviewed; no raw data or harness was found published with the post.
Five runs per condition; Microsoft scopes each result to the agent profile it measured.
One upgrade task (SPFx 1.21.1 to 1.22.2).

For designers

Installing another documentation source does not mean the agent will use it. Check whether a tool loaded and was called before you credit it with a change.

FIND-06: There are few enough tools to choose from

Cite this record

Cite the original source for any number, and keep the evidence class and model set with the figure. To point at this record, use "AX evidence register, EV-0030" and this page's address, https://agentexperience.tech/evidence/ev-0030/. The record is also in /evidence.json. The register's licence will be confirmed before its source repository is published.