From Quantity to Quality: A Risk-Based Approach to AI-Assisted QA at Scale

Description and use

A risk‑based approach using AI analyzes release changes to identify high‑impact areas and prioritize tests, cutting regression time and improving quality.

Problem

Bugs still reach production despite large test suites, and regression testing takes too long, causing delays and wasted effort.

Outcome

The approach cut regression time by roughly half, focused effort on high‑impact changes, and avoided unnecessary test execution, though initial iterations increased effort before stabilizing.

Workflow

• 1. Collect full release branch diff, ticket descriptions, and existing test‑case mapping. 2. Feed data into AI model for impact analysis. 3. Generate report highlighting risk areas and test triage. 4. Execute must‑test and recommended tests; skip safe‑to‑skip. 5. Review report for cross‑cutting or hidden dependencies.
• 6. Iterate on input quality and model tuning over several releases.

Evidence

AI impact analysis inserted into the pre‑merge job produced a report in fifteen to twenty minutes flagging the two areas that actually mattered: a rewritten transfer flow and a changed card issuance fee calculation.

Source

Roughly 200 of the four hundred cases (50%) came back must‑test, given how wide the blast radius was.

Source