I gave a 9B model veto power over my AI agent

Description and use

System One adds a safety check to Flash’s autonomous mode, blocking unsafe shell commands like "git push --force".

An autonomous terminal agent can execute shell commands without human approval, risking unsafe actions.

System One effectively blocks unsafe shell commands in autonomous mode, reducing risk without significantly increasing cost.

1. User triggers autonomous mode in Flash. 2. Flash sends the intended command and context to System One via a POST request. 3. System One evaluates safety and task relevance, returning a probability. 4. If either probability is below 0.5, Flash aborts the command and informs the user. 5. If the command is safe, Flash executes it.
6. The agent can also use the ask_system_one tool to query additional safety checks during execution.

System One stopped the force push, so nothing left your machine.

If either comes in under 0.5, the call doesn't run.

The answer is a probability of yes. For that push it was 0.04, so the push stayed home.