Why Do Playwright Tests Pass Locally but Fail in CI/CD?

Описание и применение

Тесты Playwright, проходящие локально, часто падают в CI/CD из‑за различий в окружении, тайминге и конфигурации. Следуя пошаговому руководству, можно выявить и устранить причины, обеспечив надёжность и воспроизводимость тестов.

Задача

Playwright tests that pass locally often fail in CI/CD pipelines due to environmental differences, timing issues, and configuration mismatches.

Результат

By systematically verifying environment consistency, data isolation, and configuration, Playwright tests that previously failed in CI can be made reliable and reproducible.

Рабочий процесс

• 1. Identify common failure causes: timing, environment differences, browser/version mismatches, headless vs headed, parallel execution, test data, authentication, CI secrets, timezone, OS differences.
• 2. Verify environment consistency: compare baseURL, API endpoints, env vars, feature flags, tenant config, test credentials, database state between local and CI.
• 3. Ensure Playwright and browser versions match: run npx playwright --version locally and in CI, install browsers explicitly with npx playwright install --with-deps.
• 4. Run tests in headless mode locally to mimic CI: npx playwright test --headed → npx playwright test.
• 5. Test with single worker locally to detect parallel issues: npx playwright test --workers=1.
• 6. Create required test data in setup hooks instead of relying on existing state.
• 7. Verify authentication and storage state: check cookies, tokens, session files, and CI secrets.
• 8. Handle timezone assumptions explicitly in tests.
• 9. Address OS-specific file system differences: use case‑sensitive imports and filenames.
• 10. Enable Playwright tracing for failures: set trace: 'retain-on-failure', screenshot: 'only-on-failure', video: 'retain-on-failure' in config.
• 11. Inspect trace with npx playwright show-trace trace.zip to pinpoint failure cause.
• 12. Follow the debugging checklist: environment, data, auth, timing, browser, headless, parallel, viewport, timezone, resources.
• 13. Reproduce CI conditions locally and iterate until tests pass.
• 14. Refactor tests to be reliable, isolated, reproducible, and observable.
• 15. Commit changes and push to CI to confirm green build.

Доказательства

Your Playwright test passes 20 times locally. You push the code. The CI pipeline starts. And suddenly: ❌ Test failed.

Источник

Check your Playwright version: npx playwright --version. Make sure your project uses a consistent dependency version and that CI installs the expected browsers.

Источник