Should an AI Tutor Just Give You the Answer? I Tested It

Description and use

<p>I built an AI tutor for SQL that won't hand you the answer straight away. Then I tested it against a normal version that does.</p> <p>Six people practised with one version or the other. A week later, they took a 12-question SQL test with no tutor at all.</p> <ul> <li>The <strong>"try first" group</strong> got all of their answers right: 36 of 36 (3 people × 12 questions).</li> <li>The <strong>"give me the answer" group</strong> got 25 of 36.</li> </ul> <p>That looks like a clear win. But when I presented it at my course's Demo Day, I told the room I don't credit the win to my tutor.</p> <p>

Problem

<p>I built an AI tutor for SQL that won't hand you the answer straight away. Then I tested it against a normal version that does.</p> <p>Six people practised with one version or the other. A week later, they took a 12-question SQL test with no tutor at all.</p> <ul> <li>The <strong>"try first" group</strong> got all of their answers right: 36 of 36 (3 people × 12 questions).</li> <li>The <strong>"give me the answer" group</strong> got 25 of 36.</li> </ul> <p>That looks like a clear win. But when I presented it at my course's Demo Day, I told the room I don't credit the win to my tutor.</p> <p>

Outcome

<p>I built an AI tutor for SQL that won't hand you the answer straight away. Then I tested it against a normal version that does.</p> <p>Six people practised with one version or the other. A week later, they took a 12-question SQL test with no tutor at all.</p> <ul> <li>The <strong>"try first" group</strong> got all of their answers right: 36 of 36 (3 people × 12 questions).</li> <li>The <strong>"give me the answer" group</strong> got 25 of 36.</li> </ul> <p>That looks like a clear win. But when I presented it at my course's Demo Day, I told the room I don't credit the win to my tutor.</p> <p>

Evidence

Should an AI Tutor Just Give You the Answer? I Tested It

Source