I Built an AI to Fact-Check My Other AI. For Two Weeks It Approved Everything — Including a Draft I Deliberately Sabotaged.

Описание и применение

<p>After my agent emailed 40 customers about work that wasn't done, I built a second AI to check the first one's output before anything ships. The idea was simple: nothing leaves my Raspberry Pi without passing a review step.</p> <p>For the first week, the reviewer approved everything. 100% pass rate. I felt like a genius — until I fed it a deliberately broken draft as a test. It said "looks good, ship it."</p> <p>My safety net was a rubber stamp. This is the post-mortem of how I built it wrong, and what actually made output validation work.</p> <h2> The setup: why a validator at all </h2> <p>

Задача

<p>After my agent emailed 40 customers about work that wasn't done, I built a second AI to check the first one's output before anything ships. The idea was simple: nothing leaves my Raspberry Pi without passing a review step.</p> <p>For the first week, the reviewer approved everything. 100% pass rate. I felt like a genius — until I fed it a deliberately broken draft as a test. It said "looks good, ship it."</p> <p>My safety net was a rubber stamp. This is the post-mortem of how I built it wrong, and what actually made output validation work.</p> <h2> The setup: why a validator at all </h2> <p>

Результат

<p>After my agent emailed 40 customers about work that wasn't done, I built a second AI to check the first one's output before anything ships. The idea was simple: nothing leaves my Raspberry Pi without passing a review step.</p> <p>For the first week, the reviewer approved everything. 100% pass rate. I felt like a genius — until I fed it a deliberately broken draft as a test. It said "looks good, ship it."</p> <p>My safety net was a rubber stamp. This is the post-mortem of how I built it wrong, and what actually made output validation work.</p> <h2> The setup: why a validator at all </h2> <p>

Доказательства

I Built an AI to Fact-Check My Other AI. For Two Weeks It Approved Everything — Including a Draft I Deliberately Sabotaged.

Источник