Playwright Python AI at Scale: 10 Failure Modes Every SDET Should Know How to Fix

Description and use

Implementing a locator contract and enforcing it in CI reduces locator‑related test failures to 10 % or less.

Problem

Dynamic locators and unstable DOM cause frequent test failures in large Playwright Python test suites.

Outcome

Adopting a contract‑based locator strategy and enforcing it via meta tests reduced locator‑related failures by over 90% and improved test suite maintainability.

Workflow

• 1. Define a test‑ID contract: frontend emits data‑qa attributes for interactive elements.
• 2. Configure Playwright to use data‑qa as the test‑ID attribute in conftest.py.
• 3. Adopt a locator priority ladder: role > test_id > label/placeholder > scoped CSS > disallow XPath and positional selectors.
• 4. Scope locators to component roots so row identity derives from business data, not DOM position.
• 5. Use lazy locators and filter(has=…) to maintain stability across re‑renders and re‑ordering.
• 6. Add a meta test that scans the suite for forbidden locator patterns and fails the PR if found.
• 7. Review and lint code to enforce the locator policy.
• 8. Run tests in CI with pytest‑xdist for parallel execution.
• 9. Monitor test results and iterate on locator contracts as the UI evolves.

Evidence

A design‑system upgrade breaks 300 tests in one PR, spread across 12 repositories with 12 different owners.

Source

Locator priority ladder (enforced by review and lint): 1. get_by_role() 2. get_by_test_id() 3. get_by_label()/placeholder 4. locator() with CSS 5. XPath banned.

Source

Meta test that guards the framework itself fails the PR if forbidden locators are used.

Source