Make Test Email Automation Observable

Описание и применение

Добавление простого слоя наблюдаемости к тестам, отправляющим письма, позволяет преобразовать неопределённые тайм-ауты в понятные сообщения об ошибках, облегчая отладку и повышая надёжность CI.

Задача

Email-driven tests often fail with vague timeouts, hiding the real cause such as queue delays, wrong recipients, or mailbox issues.

Результат

Adding a lightweight observability layer around email tests transforms vague timeouts into actionable diagnostics, reducing flaky failures and improving developer confidence.

Рабочий процесс

• Generate a unique run_id and log it with the test name
• Create or reserve a mailbox lease associated with the run_id
• Mark state as "requested" when the application accepts the action
• Wait for the email in the mailbox and mark state as "message_seen" with message_id
• Extract the verification code and assert it matches the expected value, marking state as "asserted"
• On timeout or error, record the last state, elapsed time, and a safe diagnostic
• Cleanup the mailbox lease after success or failure
• Ensure subsequent retries cannot read previous run messages

Доказательства

Email-driven tests often fail in the least helpful way: the assertion says that a verification message never arrived, while the real problem could be a queue delay, a wrong recipient, an expired mailbox, or a selector that stopped matching the UI.

Источник

Give every run an identity, record a few meaningful state changes, and make the final failure explain what happened.

Источник

The run_id is the thread that ties application logs, test output, and mailbox checks together.

Источник

Treat a test mailbox as a lease, not as an identity. Create it for one run, associate it with the run_id, and release it when the test finishes.

Источник