When AI essay feedback misquotes a student, it is mostly in short quotes: both of my predictions reversed again

Description and use

Short quotes in AI feedback are more likely to be genuine misquotes, while long quotes tend to be reworded passages, guiding better verification of model accuracy.

Problem

AI essay feedback often places students’ words in quotation marks, but it is unclear whether these quoted passages truly reflect the student’s original text.

Outcome

Short quoted passages (4‑6 words) are more likely to contain altered words and to be loose near‑misses, whereas long quoted passages (11+ words) rarely alter words and are often reworded passages. The fixed similarity cutoff does not explain the observed reversal of predictions.

Workflow

• 1. Collect 8,946 IELTS essays and their GLM‑4.7 feedback.
• 2. Identify all quoted passages in the feedback.
• 3. Compute word‑level similarity between each quote and the original essay.
• 4. Classify quotes as "tight" (similarity ≥0.8) or "loose" (similarity <0.8).
• 5. Split tight quotes by length (4‑6 words vs. 11+ words).
• 6. Calculate the proportion of tight quotes that alter a word for each length group.
• 7. Calculate the proportion of changed quotes that are loose for each length group.
• 8. Compare observed proportions to two pre‑made predictions.
• 9. Interpret results and recommend verification guidelines.

Evidence

About half of all changed quotes were loose.

3.19% of tight 4-6-word quotes alter a word, against 0.74% of tight quotes of 11 or more words.

50.1% of changed 4-6-word quotes are loose, against 66.3% of changed quotes of 11 or more words.

Everything is open: the plan, code, tests and counts are at https://github.com/munimtechnologies/feedback-quote-audit

Source