I gave AI agents a harm budget instead of trust and ran 19 experiments

Description and use

A harm-budget approach to AI agent trust reduced financial loss and improved autonomy control.

Developers and customers lacked a reliable way to assess whether an AI agent could be trusted to act autonomously without causing financial loss.

Agents with orchestrators averaged $470 profit per run versus $5,750 loss without; harm budget enforcement reduced total harm and improved trust.

1. Design agent passport and orchestrator to monitor actions
2. Define hidden exam tasks with known correct answers and associated monetary harm per mistake
3. Set a harm budget per period and calculate allowed autonomous actions
4. Run sandbox experiments with fake shop, customers, and money
5. Compare agent performance with and without orchestrator
6. Analyze results, adjust orchestrator behavior, and refine harm calculations

I care how much money each action the agent takes on its own loses on average.

Agents that learned from someone else's checked experience reached full autonomy by round 5, agents learning alone took until round 7 or 9.

The model costs about $0.002 per ticket. Paid checks, human time and mostly harm make up more than 99 percent of the cost.