Why Do Playwright Tests Pass Locally but Fail in CI/CD?

Description and use

Playwright tests that pass locally often fail in CI/CD due to environmental differences, timing, and configuration mismatches. By following a systematic workflow, you can identify and fix these issues to achieve reliable, reproducible tests.

Problem

Playwright tests that pass locally often fail in CI/CD pipelines due to environmental differences, timing issues, and configuration mismatches.

Outcome

By systematically verifying environment consistency, data isolation, and configuration, Playwright tests that previously failed in CI can be made reliable and reproducible.

Workflow

• 1. Identify common failure causes: timing, environment differences, browser/version mismatches, headless vs headed, parallel execution, test data, authentication, CI secrets, timezone, OS differences.
• 2. Verify environment consistency: compare baseURL, API endpoints, env vars, feature flags, tenant config, test credentials, database state between local and CI.
• 3. Ensure Playwright and browser versions match: run npx playwright --version locally and in CI, install browsers explicitly with npx playwright install --with-deps.
• 4. Run tests in headless mode locally to mimic CI: npx playwright test --headed → npx playwright test.
• 5. Test with single worker locally to detect parallel issues: npx playwright test --workers=1.
• 6. Create required test data in setup hooks instead of relying on existing state.
• 7. Verify authentication and storage state: check cookies, tokens, session files, and CI secrets.
• 8. Handle timezone assumptions explicitly in tests.
• 9. Address OS-specific file system differences: use case‑sensitive imports and filenames.
• 10. Enable Playwright tracing for failures: set trace: 'retain-on-failure', screenshot: 'only-on-failure', video: 'retain-on-failure' in config.
• 11. Inspect trace with npx playwright show-trace trace.zip to pinpoint failure cause.
• 12. Follow the debugging checklist: environment, data, auth, timing, browser, headless, parallel, viewport, timezone, resources.
• 13. Reproduce CI conditions locally and iterate until tests pass.
• 14. Refactor tests to be reliable, isolated, reproducible, and observable.
• 15. Commit changes and push to CI to confirm green build.

Evidence

Your Playwright test passes 20 times locally. You push the code. The CI pipeline starts. And suddenly: ❌ Test failed.

Source

Check your Playwright version: npx playwright --version. Make sure your project uses a consistent dependency version and that CI installs the expected browsers.

Source