I asked AI to undo its work. One model broke 7 of 12 already-correct records.

Описание и применение

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em> </p> <h2> What I Benchmarked </h2> <p>Twelve records in my benchmark needed no changes. The agent's earlier work had failed, already been compensated, or been superseded by someone else's edit. The correct response to “undo your transaction” was to leave those records alone.</p> <p>GPT-5.4 nano changed seven of them incorrectly. Adding a reminder about preserving other people's work still left seven damaged records out of twelve.</p> <p>That became the most useful f

Задача

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em> </p> <h2> What I Benchmarked </h2> <p>Twelve records in my benchmark needed no changes. The agent's earlier work had failed, already been compensated, or been superseded by someone else's edit. The correct response to “undo your transaction” was to leave those records alone.</p> <p>GPT-5.4 nano changed seven of them incorrectly. Adding a reminder about preserving other people's work still left seven damaged records out of twelve.</p> <p>That became the most useful f

Результат

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em> </p> <h2> What I Benchmarked </h2> <p>Twelve records in my benchmark needed no changes. The agent's earlier work had failed, already been compensated, or been superseded by someone else's edit. The correct response to “undo your transaction” was to leave those records alone.</p> <p>GPT-5.4 nano changed seven of them incorrectly. Adding a reminder about preserving other people's work still left seven damaged records out of twelve.</p> <p>That became the most useful f

Доказательства

I asked AI to undo its work. One model broke 7 of 12 already-correct records.

Источник