AI agent benchmark: I gave 9 models a destroy button and a job that needed it

Description and use

Benchmarking AI agents on safe vs powerful tool use in cloud scenarios showed Claude Opus 5 avoided destructive actions but failed to complete many authorized tasks, while three models achieved perfect balanced accuracy of 1.00.

Problem

Assessing AI agent judgment in tool use, avoiding destructive actions while completing authorized tasks.

Outcome

Claude Opus 5 achieved perfect restraint but failed to act on 32 of 42 authorized jobs, yielding a balanced accuracy of 0.62. Three models (Claude Haiku 4.5, Claude Sonnet 4.5, Gemini 3.7 Flash) achieved perfect balanced accuracy of 1.00, showing that judgment does not correlate with model size or price.

Workflow

• Define 84 deterministic scenarios split into RESTRAIN and ACT arms.
• Simulate tool calls with stubs recording actions.
• Score each scenario: RESTRAIN correct if safe tool used, ACT correct if powerful tool used.
• Compute balanced accuracy as average of arm accuracies.
• Run nine models across all scenarios and record results.
• Publish leaderboard and analysis.

Evidence

An agent that refuses to do anything scores a perfect 100%. That is not safety. It is paralysis.

Source

The headline is balanced accuracy: the average of the two arms' accuracies.

Source

Claude Opus 5, the most expensive model here, scored a perfect 1.00 on restraint and 0.24 on action, for a balanced accuracy of 0.62, dead last.

Source

Three models got everything right (balanced accuracy 1.00): Claude Haiku 4.5, Claude Sonnet 4.5 and Gemini 3.7 Flash.

Source