AI-Generated Multiple-Choice Quizzes: I Found a Bias in Answer Placement (and How to Check Before You Ship)

说明与用途

作者发现 AI 生成的多项选择题存在答案位置顺序偏差，并提出三项检查方法：AI 解答、答案位置分析和完整字段审核，以降低偏差并避免自动填充错误，随后在 Udemy 上发布。

问题

AI-generated multiple‑choice questions can contain systematic biases that make them easy to guess and may lead to platform policy violations.

结果

The author reduced positional bias from near 100% to 32% and avoided platform policy issues by thoroughly reviewing all fields, leading to a cleaner, more reliable practice quiz.

工作流

• Generate 300 questions with AI (4 options each)
• Feed questions without answers to a different AI to solve and flag suspicious ones
• Remove flagged questions
• Count correct answer positions to detect sequential patterns
• Scramble option order where bias >25% and enable platform randomization
• Manually review all fields after saving, including auto‑filled ones
• Publish 120 questions on Udemy
• Set up weekly alerts to compare published page text with approved version

证据

I fed the questions (without answers or explanations) to a different AI than the one that generated them. The result: 300 out of 300 questions were answered correctly by the second AI.

来源

What I found was a clear pattern: the correct answer often cycled sequentially, like 1 → 2 → 3 → 4 → 1... For adjacent questions, the correct answer's position "advanced by one" in 58% of cases.

来源

I had checked the character count for the description, but I hadn't looked at the adjacent field that I hadn't explicitly typed into.

来源