I told two AI models to copy the student's words exactly. Their quotes did not get more exact

Description and use

Adding an explicit "copy exactly" instruction did not improve quote accuracy in small local LLMs; model choice mattered more.

Problem

Ensuring AI feedback quotes student text exactly when prompted

Outcome

Adding an explicit "copy exactly" line had no clear effect; model choice was the main determinant of quote accuracy.

Workflow

• 1. Preregistered a question about adding an explicit line before the student's text
• 2. Ran eight experiments with two 3B models (llama3.2:3b, qwen2.5:3b) on 120 essays
• 3. Collected quote matches and calculated exactness percentages
• 4. Compared results with and without the explicit line
• 5. Analyzed model-specific performance and language effects
• 6. Reported findings and provided a quote-check tool

Evidence

With the extra line: 905 of 962 matched quotes exact (94.1%).

Without it: 923 of 979 exact (94.3%).

llama3.2:3b quoted exactly about 98-99% of the time, in English and in Spanish, with or without the line.

qwen2.5:3b quoted exactly about 88-90% of the time, again with or without it.

Only 63 of the 120 planned essays were completed in all eight runs (time budgets and failed requests), though the preregistered floor of 300 matched quotes per condition was met three times over.