We Scored 17 Real Playwright Test Suites: The State of Playwright Quality in 2026

Описание и применение

Исследование оценило 17 публичных наборов тестов Playwright, выявив 5 467 нарушений правил, большинство из которых связаны с использованием сырых селекторов. Результаты показывают, что даже высоко оцененные наборы могут скрывать плотные группы анти‑паттернов.

Задача

Managed QA vendors claim to write good Playwright tests, but claims are not measurable and many real-world suites contain anti‑patterns that reduce reliability.

Результат

The study quantified 5,467 rule violations across 1,214 spec files, highlighting raw locators and missing assertions as the most common issues. Scores ranged from 77/100 to 98/100, showing that high grades can mask dense anti‑pattern clusters.

Рабочий процесс

• 1. Install playwright‑score via npm. 2. Clone each target repo’s Playwright subdirectory with a shallow, sparse git clone. 3. Run playwright-score with the standard profile and threshold 80. 4. Collect the JSON findings and aggregate by rule. 5. Analyze the results to identify high‑impact anti‑patterns.
• 6. Prioritize fixes for raw locators, hard‑coded waits, and missing assertions.
• 7. Re‑run the scorer to verify improvements.
• 8. Repeat periodically to track quality over time.

Доказательства

5,467 individual rule violations found

Источник

30.1% of all locators are raw CSS/XPath

Источник

12 / 17 repos have a test with zero assertions

Источник