Why Do Playwright Tests Pass Locally but Fail in CI/CD?

说明与用途

Playwright 测试在本地通过但在 CI/CD 中失败，原因是环境、时序和配置差异。通过系统化的工作流程，可以定位并解决问题，确保测试可靠可复现。

问题

Playwright tests that pass locally often fail in CI/CD pipelines due to environmental differences, timing issues, and configuration mismatches.

结果

By systematically verifying environment consistency, data isolation, and configuration, Playwright tests that previously failed in CI can be made reliable and reproducible.

工作流

• 1. Identify common failure causes: timing, environment differences, browser/version mismatches, headless vs headed, parallel execution, test data, authentication, CI secrets, timezone, OS differences.
• 2. Verify environment consistency: compare baseURL, API endpoints, env vars, feature flags, tenant config, test credentials, database state between local and CI.
• 3. Ensure Playwright and browser versions match: run npx playwright --version locally and in CI, install browsers explicitly with npx playwright install --with-deps.
• 4. Run tests in headless mode locally to mimic CI: npx playwright test --headed → npx playwright test.
• 5. Test with single worker locally to detect parallel issues: npx playwright test --workers=1.
• 6. Create required test data in setup hooks instead of relying on existing state.
• 7. Verify authentication and storage state: check cookies, tokens, session files, and CI secrets.
• 8. Handle timezone assumptions explicitly in tests.
• 9. Address OS-specific file system differences: use case‑sensitive imports and filenames.
• 10. Enable Playwright tracing for failures: set trace: 'retain-on-failure', screenshot: 'only-on-failure', video: 'retain-on-failure' in config.
• 11. Inspect trace with npx playwright show-trace trace.zip to pinpoint failure cause.
• 12. Follow the debugging checklist: environment, data, auth, timing, browser, headless, parallel, viewport, timezone, resources.
• 13. Reproduce CI conditions locally and iterate until tests pass.
• 14. Refactor tests to be reliable, isolated, reproducible, and observable.
• 15. Commit changes and push to CI to confirm green build.

证据

Your Playwright test passes 20 times locally. You push the code. The CI pipeline starts. And suddenly: ❌ Test failed.

来源

Check your Playwright version: npx playwright --version. Make sure your project uses a consistent dependency version and that CI installs the expected browsers.

来源