I gave a 9B model veto power over my AI agent

说明与用途

System One 为 Flash 的自主模式添加了安全检查，阻止了诸如 "git push --force" 等潜在危险命令的执行。

问题

An autonomous terminal agent can execute shell commands without human approval, risking unsafe actions.

结果

System One effectively blocks unsafe shell commands in autonomous mode, reducing risk without significantly increasing cost.

工作流

• 1. User triggers autonomous mode in Flash. 2. Flash sends the intended command and context to System One via a POST request. 3. System One evaluates safety and task relevance, returning a probability. 4. If either probability is below 0.5, Flash aborts the command and informs the user. 5. If the command is safe, Flash executes it.
• 6. The agent can also use the ask_system_one tool to query additional safety checks during execution.

证据

System One stopped the force push, so nothing left your machine.

来源

If either comes in under 0.5, the call doesn't run.

来源

The answer is a probability of yes. For that push it was 0.04, so the push stayed home.

来源