Playwright Tests Need an Evidence Ledger

说明与用途

在 Playwright 测试中实现证据账本，可精确关联邮件请求与响应，消除模糊的超时问题，提升 CI 的可靠性。

问题

Email verification tests often fail due to timing issues, but the failures hide the real problem: tests lack a record of which message they expected and why it was selected.

结果

Implementing an evidence ledger transforms ambiguous timing failures into actionable diagnostics, reduces flaky tests, and improves maintainability of email verification workflows.

工作流

• Assign a unique run ID before creating an account.
• Record the input email, request time, polling selector, and message ID returned by the inbox adapter.
• Use a correlation key (header or plus‑address) to match the message to the request.
• Poll with a deadline, interval, and typed failure reasons (e.g., message_not_found, provider_unavailable, invalid_correlation).
• Attach the in‑memory evidence array to the test report when the test ends.
• Print a compact ledger summary next to the Playwright trace on failure.
• Ensure cleanup records success or explains failure to avoid contaminating subsequent tests.

证据

Email verification tests often get labeled as timing problems. The test waits for a message, the message arrives, and the assertion still fails.

来源

An evidence ledger gives each Playwright run a small, inspectable history. It connects the signup action, the email request, the message that was observed, and the final assertion.

来源

If message-observed has no email-requested event, the test used stale evidence.

来源