Once the token limit was raised, AI feedback changed English learners' words more often

Description and use

<p><em>Made with AI agents working in my name. Code Apache-2.0, text CC BY 4.0. Only counts are published: no essay, code, hint or feedback text.</em></p> <p>When AI feedback quotes a student, the quote should be what the student wrote. If the model quietly fixes a word while "quoting", the student is shown a sentence they never wrote, and their real mistake goes unseen. I have been auditing this with preregistered tests. Two new results are in.</p> <h2> Audit 7: English learners' words were changed more often </h2> <p>Audit 6 asked whether a small open model (Qwen2.5-1.5B, on CPU) quotes Engl

Problem

<p><em>Made with AI agents working in my name. Code Apache-2.0, text CC BY 4.0. Only counts are published: no essay, code, hint or feedback text.</em></p> <p>When AI feedback quotes a student, the quote should be what the student wrote. If the model quietly fixes a word while "quoting", the student is shown a sentence they never wrote, and their real mistake goes unseen. I have been auditing this with preregistered tests. Two new results are in.</p> <h2> Audit 7: English learners' words were changed more often </h2> <p>Audit 6 asked whether a small open model (Qwen2.5-1.5B, on CPU) quotes Engl

Outcome

<p><em>Made with AI agents working in my name. Code Apache-2.0, text CC BY 4.0. Only counts are published: no essay, code, hint or feedback text.</em></p> <p>When AI feedback quotes a student, the quote should be what the student wrote. If the model quietly fixes a word while "quoting", the student is shown a sentence they never wrote, and their real mistake goes unseen. I have been auditing this with preregistered tests. Two new results are in.</p> <h2> Audit 7: English learners' words were changed more often </h2> <p>Audit 6 asked whether a small open model (Qwen2.5-1.5B, on CPU) quotes Engl

Evidence

Once the token limit was raised, AI feedback changed English learners' words more often

Source