Outside the Assertion: The Retry Budget a Pairing Hour Refused to Inline

说明与用途

在配对会议中，GET /orders/:id 的测试偶尔在重新运行后通过，导致对测试的可靠性产生疑问。决定将测试仅运行一次，任何重试都作为单独的命令执行，并记录种子、命令、退出码和输出哈希。

问题

A test for a service endpoint intermittently passes after a rerun, raising uncertainty about flakiness and the reliability of the test harness.

结果

The pairing session concluded with a documented rule that separates test execution from retries, ensuring reproducibility and traceability of failures.

工作流

• 1. Run the test once and record the result.
• 2. If a retry is needed, execute a separate command with a defined retry budget.
• 3. Record seed, command, exit code, and hash of output for each attempt.
• 4. Ensure a second process reproduces the failure before code changes.

证据

The contract check failed once, then passed on a local rerun.

来源

The senior asked three questions before anyone edited the helper.

来源

The test process runs the check once.

来源