A Change That Does Nothing Should Score Zero. Mine Didn't.

Описание и применение

Контрольные проверки, где ожидается известный результат (например, ноль), выявили скрытые ошибки измерений, гарантируя достоверность последующих данных.

Задача

Measurement pipelines can silently produce plausible but incorrect numbers, leading to unverified results.

Результат

Implementing a simple control that verifies a no-op scores zero uncovered hidden measurement bugs, ensuring that subsequent results are trustworthy.

Рабочий процесс

• Run a pilot test to identify bugs in measurement code.
• Identify control cases where the expected outcome is known (e.g., no-op should score zero).
• Feed the control input to the pipeline and verify the output matches the expected value.
• If the output differs, investigate and fix measurement code.
• Repeat control checks before proceeding with the main experiment.
• Use controls to confirm instrument reliability throughout the study.
• Publish findings only after confirming controls pass.

Доказательства

Build controls whose answer you already know.

Источник

A change that does nothing should always score zero.

Источник

Mine did not score zero. More than once.

Источник

Before you trust a number about someone else's system, make your instrument prove it can produce a number it already knows.

Источник