From experienced tester to AI testing: the roadmap, in order

Description and use

<p><strong>Short answer:</strong> learn it in this order — the non-determinism mindset, prompts as test inputs, evals (the new assertion), LLM/RAG failure modes, then verifying AI-written tests with mutation testing. Skip the math-heavy ML theory; you're testing behavior, not training models.</p> <p>I keep seeing the same post on r/softwaretesting: <em>"10 years of manual and automation under my belt, and the AI stuff makes me feel like a junior again. Where do I even start?"</em> The honest answer is that you're closer than you think. Maybe 70% of what makes you a good tester transfers direct

Problem

<p><strong>Short answer:</strong> learn it in this order — the non-determinism mindset, prompts as test inputs, evals (the new assertion), LLM/RAG failure modes, then verifying AI-written tests with mutation testing. Skip the math-heavy ML theory; you're testing behavior, not training models.</p> <p>I keep seeing the same post on r/softwaretesting: <em>"10 years of manual and automation under my belt, and the AI stuff makes me feel like a junior again. Where do I even start?"</em> The honest answer is that you're closer than you think. Maybe 70% of what makes you a good tester transfers direct

Outcome

<p><strong>Short answer:</strong> learn it in this order — the non-determinism mindset, prompts as test inputs, evals (the new assertion), LLM/RAG failure modes, then verifying AI-written tests with mutation testing. Skip the math-heavy ML theory; you're testing behavior, not training models.</p> <p>I keep seeing the same post on r/softwaretesting: <em>"10 years of manual and automation under my belt, and the AI stuff makes me feel like a junior again. Where do I even start?"</em> The honest answer is that you're closer than you think. Maybe 70% of what makes you a good tester transfers direct

Evidence

From experienced tester to AI testing: the roadmap, in order

Source