I told two AI models to copy the student's words exactly. Their quotes did not get more exact

说明与用途

在小型本地LLM中添加“精确复制”指令并未提升引用准确性；模型选择更为重要。

问题

Ensuring AI feedback quotes student text exactly when prompted

结果

Adding an explicit "copy exactly" line had no clear effect; model choice was the main determinant of quote accuracy.

工作流

• 1. Preregistered a question about adding an explicit line before the student's text
• 2. Ran eight experiments with two 3B models (llama3.2:3b, qwen2.5:3b) on 120 essays
• 3. Collected quote matches and calculated exactness percentages
• 4. Compared results with and without the explicit line
• 5. Analyzed model-specific performance and language effects
• 6. Reported findings and provided a quote-check tool

证据

With the extra line: 905 of 962 matched quotes exact (94.1%).

Without it: 923 of 979 exact (94.3%).

llama3.2:3b quoted exactly about 98-99% of the time, in English and in Spanish, with or without the line.

qwen2.5:3b quoted exactly about 88-90% of the time, again with or without it.

Only 63 of the 120 planned essays were completed in all eight runs (time budgets and failed requests), though the preregistered floor of 300 matched quotes per condition was met three times over.