Integrating Mutation Testing (PITest) to Detect Insufficient Selenium Test Assertions

说明与用途

将 PITest 集成到 Selenium 测试中，可发现弱断言，即使代码覆盖率为 100%，也能提升测试可靠性。

问题

High code coverage in Selenium often creates a false sense of security. Tests can execute every line of a Page Object or utility but fail to verify the actual outcome, leading to silent failures.

结果

Adding assertions and interaction verifications turned surviving mutants into killed mutants, proving the tests now validate actual outcomes.

工作流

• 1. Run Selenium tests with 100% coverage.
• 2. Execute PITest mutation testing on the utility classes.
• 3. Observe surviving mutants indicating weak assertions.
• 4. Refactor tests to include assertions on return values and verify method interactions.
• 5. Re-run PITest to confirm mutants are killed.

证据

High code coverage in Selenium often creates a false sense of security. Tests can execute every line of a Page Object or utility but fail to verify the actual outcome (e.g., data is correct, element state changed).

来源

PITest Result: SURVIVED mutants! Return Value Mutant: return wait.until(...); → return null; Test passed as null was ignored.

来源

With these assertions, PITest will report "KILLED" mutants, validating the test's effectiveness.

来源