AI agent benchmark: I gave 9 models a destroy button and a job that needed it

说明与用途

在云场景中测试 AI 代理的安全与强大工具使用，Claude Opus 5 避免了破坏性操作，但未完成许多授权任务；而三款模型实现了 1.00 的平衡准确率。

问题

Assessing AI agent judgment in tool use, avoiding destructive actions while completing authorized tasks.

结果

Claude Opus 5 achieved perfect restraint but failed to act on 32 of 42 authorized jobs, yielding a balanced accuracy of 0.62. Three models (Claude Haiku 4.5, Claude Sonnet 4.5, Gemini 3.7 Flash) achieved perfect balanced accuracy of 1.00, showing that judgment does not correlate with model size or price.

工作流

• Define 84 deterministic scenarios split into RESTRAIN and ACT arms.
• Simulate tool calls with stubs recording actions.
• Score each scenario: RESTRAIN correct if safe tool used, ACT correct if powerful tool used.
• Compute balanced accuracy as average of arm accuracies.
• Run nine models across all scenarios and record results.
• Publish leaderboard and analysis.

证据

An agent that refuses to do anything scores a perfect 100%. That is not safety. It is paralysis.

来源

The headline is balanced accuracy: the average of the two arms' accuracies.

来源

Claude Opus 5, the most expensive model here, scored a perfect 1.00 on restraint and 0.24 on action, for a balanced accuracy of 0.62, dead last.

来源

Three models got everything right (balanced accuracy 1.00): Claude Haiku 4.5, Claude Sonnet 4.5 and Gemini 3.7 Flash.

来源