Email Checks Need a Failure Budget

说明与用途

邮件验证测试常因模糊的超时和重试导致失败。通过定义失败预算、记录时间戳并保存轻量级收据，测试变得可控、可诊断且在 CI 中更快。

问题

Email verification tests often fail silently due to unclear timeouts and retries, making debugging slow and CI expensive.

结果

Tests become predictable: bounded wait times, clear failure reasons, and efficient CI usage. Debugging shifts from vague timeouts to concrete evidence of queueing, delivery, and retry causes.

工作流

• 1. Define a budget object with totalMs, pollEveryMs, maxAttempts, and subject.
• 2. Record a startedAt timestamp before triggering the email.
• 3. Poll the inbox at pollEveryMs until totalMs expires, filtering messages after startedAt.
• 4. If a matching subject is found, return match info and poll count.
• 5. On timeout, throw an error with poll count.
• 6. Before retrying, save a receipt containing run ID, inbox ID, trigger timestamp, poll number, elapsed time, matching IDs, and failure reason.
• 7. Retry only if the receipt indicates a transient delivery issue; otherwise fail immediately.
• 8. Log all receipts and poll data for both passes and failures.

证据

"If those clocks are mixed together, a timeout says very little. Maybe the app never queued the message. Maybe the message arrived after the final poll. Maybe the test read an older message."

来源

"I also record a startedAt timestamp before the action that triggers the email. Every inbox query filters for messages newer than that boundary."

来源

"Before the next attempt, save a small receipt. It does not need to contain the message body or private user data."

来源