Playwright Email Tests Need a Failure Triage Map

Description and use

Playwright email tests now employ a triage map that classifies failures into four states, enabling precise diagnosis and faster root‑cause analysis.

Problem

Email verification tests often fail with vague timeouts, making it hard to identify whether the issue lies in the application, mail delivery, or test client.

Outcome

Tests now report a clear triage state, reducing noise from generic timeouts and enabling faster root‑cause analysis.

Workflow

• 1. Generate a unique correlation value for the test run.
• 2. Submit the registration request and record the timestamp.
• 3. Observe the send event in the application logs and record its timestamp.
• 4. Poll the mailbox API for a message containing the correlation value, recording each poll’s result and the number of messages seen.
• 5. If a matching message is found, record its timestamp and verify the verification link’s host and token.
• 6. Follow the link and record the verification response timestamp.
• 7. Classify the final state as Not requested, Requested not received, Received not usable, or Usable not completed.
• 8. Attach the triage state and relevant evidence (timestamps, counts, correlation) to the test report.
• 9. If the test fails, review the recorded evidence to determine the failure point before adjusting timeouts.

Evidence

Email verification tests often fail in a confusing way: the UI says “check your inbox,” the message exists somewhere, and the test still times out.

Source

I use four states in the test report: Not requested, Requested, not received, Received, not usable, Usable, not completed.

Source

Give each test a correlation value and carry it through the request, the message subject, and the assertion.

Source

Here is a small polling helper. It waits for a message matching a correlation value, but it also records the reason for each unsuccessful poll.

Source