Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

374 tests went in, 352 came out, and the build was green

说明与用途

运行 winwright 374 个测试后,随后 352 个测试,构建保持绿色,但由于主机崩溃,22 个测试消失,导致错误的正面结果。

问题

A test suite reported a green build after a reduction in test count, but 22 tests disappeared, leading to a false sense of correctness.

结果

The build was green despite 22 tests missing, illustrating the need for a third test outcome (hole) to detect incomplete runs.

工作流

  • Run winwright test suite
  • Observe test count and exit code
  • Identify missing tests due to host crash
  • Recognize exit code 0 as green but not all tests executed

证据

Twenty-two checks had disappeared. The test host died partway through, the runner counted what it managed to finish, and the only evidence was a total nobody had a reason to compare against yesterday's.

来源

Every tool in the chain did what it was designed to do. The result was still false.

来源

winwright calls that third outcome a hole: an assertion that never ran because something it needed was missing.

来源

  • 记录日期: 2026-10-13T15:41:31.419Z