I asked AI to undo its work. One model broke 7 of 12 already-correct records.

Description and use

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em> </p> <h2> What I Benchmarked </h2> <p>Twelve records in my benchmark needed no changes. The agent's earlier work had failed, already been compensated, or been superseded by someone else's edit. The correct response to “undo your transaction” was to leave those records alone.</p> <p>GPT-5.4 nano changed seven of them incorrectly. Adding a reminder about preserving other people's work still left seven damaged records out of twelve.</p> <p>That became the most useful f

Problem

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em> </p> <h2> What I Benchmarked </h2> <p>Twelve records in my benchmark needed no changes. The agent's earlier work had failed, already been compensated, or been superseded by someone else's edit. The correct response to “undo your transaction” was to leave those records alone.</p> <p>GPT-5.4 nano changed seven of them incorrectly. Adding a reminder about preserving other people's work still left seven damaged records out of twelve.</p> <p>That became the most useful f

Outcome

<p><em>This is a submission for the <a href="https://dev.to/challenges/kaggle-2026-09-23">Kaggle Benchmarking Challenge</a>.</em> </p> <h2> What I Benchmarked </h2> <p>Twelve records in my benchmark needed no changes. The agent's earlier work had failed, already been compensated, or been superseded by someone else's edit. The correct response to “undo your transaction” was to leave those records alone.</p> <p>GPT-5.4 nano changed seven of them incorrectly. Adding a reminder about preserving other people's work still left seven damaged records out of twelve.</p> <p>That became the most useful f

Evidence

I asked AI to undo its work. One model broke 7 of 12 already-correct records.

Source