I gave a 9B model veto power over my AI agent

Описание и применение

Система One добавляет проверку безопасности в автономный режим терминального агента Flash, блокируя потенциально опасные команды, такие как "git push --force".

Задача

An autonomous terminal agent can execute shell commands without human approval, risking unsafe actions.

Результат

System One effectively blocks unsafe shell commands in autonomous mode, reducing risk without significantly increasing cost.

Рабочий процесс

• 1. User triggers autonomous mode in Flash. 2. Flash sends the intended command and context to System One via a POST request. 3. System One evaluates safety and task relevance, returning a probability. 4. If either probability is below 0.5, Flash aborts the command and informs the user. 5. If the command is safe, Flash executes it.
• 6. The agent can also use the ask_system_one tool to query additional safety checks during execution.

Доказательства

System One stopped the force push, so nothing left your machine.

Источник

If either comes in under 0.5, the call doesn't run.

Источник

The answer is a probability of yes. For that push it was 0.04, so the push stayed home.

Источник