I wrote that their test harness had no denominator. Their own script header said it first.

Описание и применение

В тестовом наборе регрессий отсутствовали фикстуры, что приводило к тихому дрейфу. После обновления скрипта и рабочего процесса все фикстуры теперь обнаруживаются и проверяются, и отсутствие фикстуры приводит к ошибке сборки.

Задача

A regression harness silently ignored missing fixtures, causing undetected drift and inaccurate test coverage.

Результат

The harness now fails early when fixtures are missing, ensuring all tests run as intended and preventing silent drift.

Рабочий процесс

• 1. Inspect regression harness for console warnings.
• 2. Locate the script that determines which fixtures run (plan-regression-shards.mjs).
• 3. Read the script header to understand fixture scheduling rules.
• 4. Verify the workflow that calls the script and generates the shard matrix.
• 5. Run the script locally to confirm it fails when a fixture is missing.
• 6. Update the fixture schedule (shard-schedule.json) to include all valid fixtures and exclude invalid ones with reasons.
• 7. Commit changes and run CI to ensure no missing fixtures cause failures.

Доказательства

Silent drift. A fixture only ran if someone remembered to paste its name into the YAML. 25 fixtures that the harness can run were in no shard at all, some for months, and 3 more were rejected at load time for invalid meta.json with nothing louder than a console warning.

Источник

Fixtures are now discovered from disk. Every one must be either scheduled (with a timing) or explicitly excluded with a reason, or this script fails.

Источник