I re-ran 152 versions of popular MCP servers. Nearly half the updates changed what the tools tell your agent.

Description and use

In a study of 152 MCP server releases, 47% altered the tool surface, mainly via rewritten parameter descriptions. The mcpgawk tool automates detection and blocking of such silent changes, enhancing agent safety.

Problem

MCP servers are approved once, but unpinned npm packages update silently, changing tool surfaces without re‑approval, potentially altering agent behavior.

Outcome

In a study of 152 stable MCP server releases, 47% changed the tool surface; 48% in an independent replication. Most changes were rewritten parameter descriptions (92 instances). The mcpgawk tool automates detection and enforcement of such changes.

Workflow

• 1. Pin MCP server versions in configuration to avoid automatic updates.
• 2. Snapshot the tool list (names, descriptions, schemas) upon approval.
• 3. On each launch, diff the current tool list against the snapshot, including descriptions and schemas.
• 4. If differences are found, refuse to call altered tools until the user explicitly approves the change.
• 5. Optionally probe servers without API keys to detect key‑protected tool changes.

Evidence

I took the 30 most-downloaded MCP server packages on npm. For each one I pulled every stable version published between March and September 2026 (up to 12 per package), and launched each version the same way:

Source

Updates that changed the tool surface 40 (47%)

Source

An independent researcher, using her own setup on nine servers, measured 48% across 96 releases. Two methods, two machines, the same answer: roughly every second update changes what your agent is told.

Source

Tools added / removed: 12 / 4

Source

Parameter descriptions rewritten: 92 — the biggest category by far

Source

I built mcpgawk: it records what every tool said when you approved it, and if a tool rewrites its description or gains power afterwards, the call is refused before it runs — until you decide.

Source