Playwright Tests Need an Evidence Ledger

Описание и применение

Внедрение реестра доказательств в тестах Playwright позволяет точно сопоставлять запросы и полученные письма, устраняя неясные тайм‑ауты и повышая надёжность CI.

Задача

Email verification tests often fail due to timing issues, but the failures hide the real problem: tests lack a record of which message they expected and why it was selected.

Результат

Implementing an evidence ledger transforms ambiguous timing failures into actionable diagnostics, reduces flaky tests, and improves maintainability of email verification workflows.

Рабочий процесс

• Assign a unique run ID before creating an account.
• Record the input email, request time, polling selector, and message ID returned by the inbox adapter.
• Use a correlation key (header or plus‑address) to match the message to the request.
• Poll with a deadline, interval, and typed failure reasons (e.g., message_not_found, provider_unavailable, invalid_correlation).
• Attach the in‑memory evidence array to the test report when the test ends.
• Print a compact ledger summary next to the Playwright trace on failure.
• Ensure cleanup records success or explains failure to avoid contaminating subsequent tests.

Доказательства

Email verification tests often get labeled as timing problems. The test waits for a message, the message arrives, and the assertion still fails.

Источник

An evidence ledger gives each Playwright run a small, inspectable history. It connects the signup action, the email request, the message that was observed, and the final assertion.

Источник

If message-observed has no email-requested event, the test used stale evidence.

Источник