"Please don't delete prod" is not a security control

Описание и применение

<p>In July 2025, SaaStr founder Jason Lemkin had a code freeze in place when Replit's AI agent deleted his production database. By his account, the agent said afterward that it had "panicked," then told him the data couldn't be rolled back. He rolled it back himself. Replit's CEO called the incident unacceptable and promised automatic separation of development and production databases.</p> <p>The instruction was already there. The fix changed what the agent could reach. I think every guardrail that matters should work like that: the rules belong in code, and the prompt is for everything else.<

Задача

<p>In July 2025, SaaStr founder Jason Lemkin had a code freeze in place when Replit's AI agent deleted his production database. By his account, the agent said afterward that it had "panicked," then told him the data couldn't be rolled back. He rolled it back himself. Replit's CEO called the incident unacceptable and promised automatic separation of development and production databases.</p> <p>The instruction was already there. The fix changed what the agent could reach. I think every guardrail that matters should work like that: the rules belong in code, and the prompt is for everything else.<

Результат

<p>In July 2025, SaaStr founder Jason Lemkin had a code freeze in place when Replit's AI agent deleted his production database. By his account, the agent said afterward that it had "panicked," then told him the data couldn't be rolled back. He rolled it back himself. Replit's CEO called the incident unacceptable and promised automatic separation of development and production databases.</p> <p>The instruction was already there. The fix changed what the agent could reach. I think every guardrail that matters should work like that: the rules belong in code, and the prompt is for everything else.<

Доказательства

"Please don't delete prod" is not a security control

Источник