Mailbox Leases for Reliable AI Email Tests

Description and use

AI email tests often become flaky due to shared inboxes. The solution is to give each run a temporary lease, set a message cursor, and record a failure receipt, ensuring ownership, isolation, and clear logs.

Problem

AI email tests suffer from flaky results because multiple runs share a mailbox, leading to ownership ambiguity and race conditions.

Outcome

Tests become deterministic, retries are meaningful, and failure logs contain actionable context without exposing sensitive data.

Workflow

• Acquire a mailbox lease with a TTL
• Create a cursor before sending the signup request
• Trigger the signup request using the lease address
• Wait for a message after the cursor with a subject filter
• Assert the message belongs to the run
• Open the verification URL in a browser
• Release the lease in a finally block

Evidence

A lease is simply a record that says: this run owns this mailbox until a known expiry time.

Source

The cursor matters as much as the address. If a retry asks for “the newest message,” an old token can win a race against the new one.

Source