We Scored 17 Real Playwright Test Suites: The State of Playwright Quality in 2026

Description and use

The study scored 17 public Playwright test suites, uncovering 5,467 rule violations, most of which are raw locator anti‑patterns. Even high‑scoring suites can hide dense anti‑pattern clusters.

Problem

Managed QA vendors claim to write good Playwright tests, but claims are not measurable and many real-world suites contain anti‑patterns that reduce reliability.

Outcome

The study quantified 5,467 rule violations across 1,214 spec files, highlighting raw locators and missing assertions as the most common issues. Scores ranged from 77/100 to 98/100, showing that high grades can mask dense anti‑pattern clusters.

Workflow

• 1. Install playwright‑score via npm. 2. Clone each target repo’s Playwright subdirectory with a shallow, sparse git clone. 3. Run playwright-score with the standard profile and threshold 80. 4. Collect the JSON findings and aggregate by rule. 5. Analyze the results to identify high‑impact anti‑patterns.
• 6. Prioritize fixes for raw locators, hard‑coded waits, and missing assertions.
• 7. Re‑run the scorer to verify improvements.
• 8. Repeat periodically to track quality over time.

Evidence

5,467 individual rule violations found

Source

30.1% of all locators are raw CSS/XPath

Source

12 / 17 repos have a test with zero assertions

Source