Playwright Email Tests Need a Signal Boundary

Description and use

Playwright email verification tests become flaky due to a single timeout that hides independent failures in delivery, selection, and consumption. By separating signals, using a unique run ID, and bounded timeouts, reliability and diagnostics improve.

Problem

Email verification tests become flaky because a single timeout hides independent failures in delivery, selection, and consumption steps.

Outcome

Tests become more reliable and diagnosable by separating signals, using unique correlation values, and providing bounded timeouts with detailed error output.

Workflow

• Generate a unique run ID before form submission
• Create a test email address via inbox client with the run ID
• Fill the email field and submit the form
• Wait for a verification message matching the run ID and subject
• Assert the verification URL contains "/verify"
• Navigate to the verification URL
• Assert the page shows "Email verified"

Evidence

The application may queue a message, a provider may deliver it later, and the inbox API may return messages from a previous run.

Source

The goal is not to retry everything. The goal is to know whether the message was never created, was created but not visible, or was consumed by the wrong test.

Source

A useful contract is: create a unique mailbox or test address for the run; expose a stable correlation value, such as a run ID; poll only for messages matching that value; return the message ID and verification URL as separate fields; record when the message was claimed and when the token was consumed.

Source