Email Checks Need a Failure Budget

Описание и применение

Тесты, проверяющие получение писем, часто терпят неудачу из‑за неопределённых таймаутов и повторных попыток. Введение бюджета отказов, фиксирование временных меток и создание простых чеков‑свидетельств позволяют ограничить время ожидания, сделать тесты предсказуемыми и ускорить CI.

Задача

Email verification tests often fail silently due to unclear timeouts and retries, making debugging slow and CI expensive.

Результат

Tests become predictable: bounded wait times, clear failure reasons, and efficient CI usage. Debugging shifts from vague timeouts to concrete evidence of queueing, delivery, and retry causes.

Рабочий процесс

• 1. Define a budget object with totalMs, pollEveryMs, maxAttempts, and subject.
• 2. Record a startedAt timestamp before triggering the email.
• 3. Poll the inbox at pollEveryMs until totalMs expires, filtering messages after startedAt.
• 4. If a matching subject is found, return match info and poll count.
• 5. On timeout, throw an error with poll count.
• 6. Before retrying, save a receipt containing run ID, inbox ID, trigger timestamp, poll number, elapsed time, matching IDs, and failure reason.
• 7. Retry only if the receipt indicates a transient delivery issue; otherwise fail immediately.
• 8. Log all receipts and poll data for both passes and failures.

Доказательства

"If those clocks are mixed together, a timeout says very little. Maybe the app never queued the message. Maybe the message arrived after the final poll. Maybe the test read an older message."

Источник

"I also record a startedAt timestamp before the action that triggers the email. Every inbox query filters for messages newer than that boundary."

Источник

"Before the next attempt, save a small receipt. It does not need to contain the message body or private user data."

Источник