I gave AI agents a harm budget instead of trust and ran 19 experiments

说明与用途

通过损害预算评估AI代理的信任度，降低财务损失并提升自主性控制。

问题

Developers and customers lacked a reliable way to assess whether an AI agent could be trusted to act autonomously without causing financial loss.

结果

Agents with orchestrators averaged $470 profit per run versus $5,750 loss without; harm budget enforcement reduced total harm and improved trust.

工作流

• 1. Design agent passport and orchestrator to monitor actions
• 2. Define hidden exam tasks with known correct answers and associated monetary harm per mistake
• 3. Set a harm budget per period and calculate allowed autonomous actions
• 4. Run sandbox experiments with fake shop, customers, and money
• 5. Compare agent performance with and without orchestrator
• 6. Analyze results, adjust orchestrator behavior, and refine harm calculations

证据

I care how much money each action the agent takes on its own loses on average.

来源

Agents that learned from someone else's checked experience reached full autonomy by round 5, agents learning alone took until round 7 or 9.

来源

The model costs about $0.002 per ticket. Paid checks, human time and mostly harm make up more than 99 percent of the cost.

来源