Playwright Email Tests Need a Clock Skew Matrix

Description and use

Playwright email tests frequently fail due to timing differences between the app, mailbox, and test runner. By employing a clock skew matrix, recording timestamps at each boundary, and verifying message ownership, tests become reliable and failures easier to diagnose.

Problem

Email tests often fail because the test assumes the newest email is the one just sent, but timing differences between the app, mailbox, and test runner cause flakiness.

Outcome

Tests become repeatable and failures are easier to diagnose by recording and comparing timestamps across request, queue, mailbox, and assertion boundaries.

Workflow

• Create a test user
• Submit the signup form
• Record request timestamp and ID
• Reserve or create an isolated mailbox
• Poll mailbox with waitForMessage specifying after=requestedAt, marker=runId, timeoutMs, intervalMs
• Verify message receivedAt and runId
• Navigate to verificationUrl
• Assert link consumption
• Clean up mailbox lease

Evidence

Email tests often fail in a confusing way: the application says a verification message was sent, but Playwright cannot find it yet.

Source

A clock skew matrix makes those boundaries visible and gives QA a repeatable way to test them.

Source

Keep the polling helper explicit about its deadline and ownership rules. Do not hide a five-minute retry loop inside a generic waitForEmail function.

Source

When the test fails, classify the first missing boundary rather than immediately increasing the timeout:

Source