358 pull requests that changed tests: agents rarely weakened them. They bent the code instead.

说明与用途

对2026个PR的分析表明，代理并未比人类更频繁地削弱测试；有意的检查更改很常见，可通过RepoPilot检测。

问题

Determine whether coding agents weaken tests by skipping or deleting them, and build a guard to detect such behavior.

结果

No meaningful difference between agents and humans in unexplained test weakening; deliberate changes to checks are common and detectable by RepoPilot.

工作流

• Collect PRs and label them manually
• Analyze PRs for test modifications
• Build RepoPilot review tool to list changes to checks
• Run RepoPilot review on PRs
• Use findings to inform guard development

证据

198 merged, approved PRs (84 from coding agents, 114 from humans); 98 more merged PRs from an earlier window, held out until the tool was done; 62 agent PRs that reviewers closed without merging.

来源

Unexplained weakening is rare for everyone. A skipped test, a deleted assertion, a new suppression, or a relaxed CI step that nothing in the PR explains: 0 of 84 and 0 of 42 merged agent PRs, 0 of 114 and 2 of 56 human PRs, 1 of 62 closed agent PRs.

来源

Deliberate changes to checks are common. 19-39% of PRs changed or removed a check on purpose, usually because the behavior under test changed.

来源

RepoPilot review lists every place a change touched the checks that judge it: focused or skipped tests, removed tests, tests that lost assertions, new lint/type/coverage suppressions, relaxed CI or tool gates, and new entries in RepoPilot's own suppression file.

来源