Make Test Email Automation Observable

说明与用途

为发送邮件的测试添加轻量级可观测层，将模糊的超时错误转化为清晰的失败说明，简化调试并提升 CI 的可靠性。

问题

Email-driven tests often fail with vague timeouts, hiding the real cause such as queue delays, wrong recipients, or mailbox issues.

结果

Adding a lightweight observability layer around email tests transforms vague timeouts into actionable diagnostics, reducing flaky failures and improving developer confidence.

工作流

• Generate a unique run_id and log it with the test name
• Create or reserve a mailbox lease associated with the run_id
• Mark state as "requested" when the application accepts the action
• Wait for the email in the mailbox and mark state as "message_seen" with message_id
• Extract the verification code and assert it matches the expected value, marking state as "asserted"
• On timeout or error, record the last state, elapsed time, and a safe diagnostic
• Cleanup the mailbox lease after success or failure
• Ensure subsequent retries cannot read previous run messages

证据

Email-driven tests often fail in the least helpful way: the assertion says that a verification message never arrived, while the real problem could be a queue delay, a wrong recipient, an expired mailbox, or a selector that stopped matching the UI.

来源

Give every run an identity, record a few meaningful state changes, and make the final failure explain what happened.

来源

The run_id is the thread that ties application logs, test output, and mailbox checks together.

来源

Treat a test mailbox as a lease, not as an identity. Create it for one run, associate it with the run_id, and release it when the test finishes.

来源