Make CI Email Checks Explain Their Failures

Описание и применение

В CI‑тестах проверки подтверждения email часто выдаются непонятные ошибки. Добавив структурированные доказательства (runId, inbox, requestId, таймстампы, количество сообщений и причину сбоя) и сохраняв их в логах, можно быстро определить, где именно произошла ошибка – в браузере, API, воркере или почтовом ящике.

Задача

Email verification tests are hard to debug because failures often hide the real issue behind generic timeouts, making it unclear whether the problem lies in the browser, API, worker, or inbox.

Результат

CI email checks now produce actionable evidence that distinguishes between submission, delivery, matching, and link issues, turning flaky failures into clear engineering problems.

Рабочий процесс

• Generate a unique runId and inbox address for the test
• Submit signup form and capture requestId from response
• Start polling the inbox with a deadline and interval
• On each poll, record messagesSeen and match by requestId
• If message found, record matchedMessageId and exit loop
• If timeout, set failureReason based on whether any messages were seen
• Attach the evidence JSON to CI logs and throw an informative error

Доказательства

A test can say “verification email not found” while the real problem is a delayed worker, the wrong inbox, a stale message, or a request that never reached the mail service.

Источник

Once CI can answer which inbox, which request, which messages, and how long it waited, a flaky-looking test becomes a normal engineering problem.

Источник