The Formatter Looked Clean. Property Testing Found the Missing Zeroes.

Описание и применение

Патч агента заменил форматтер, удалив паддинг нулями, но юнит‑тесты не обнаружили ошибки. С помощью модели OpenAI извлечены инварианты старого кода, преобразованные в свойства, которые выявили нарушение контракта и заблокировали слияние.

Задача

An agent patch replaced a hand‑written formatter with a shorter version that silently dropped zero‑padding, causing incorrect output for values under ten, but unit tests did not catch it.

Результат

The property gate caught a contract violation that unit tests missed, preventing a broken merge and ensuring the formatter’s zero‑padding behavior was preserved.

Рабочий процесс

• Check out the old implementation from git.
• Generate a list of invariants from the old code using a language model.
• Convert each invariant into a property test (e.g., zero‑padding for hours, minutes, seconds).
• Run the properties against both old and new implementations using Hypothesis.
• Block the merge if any property holds on the old version but fails on the new one.

Доказательства

The new formatter silently dropped zero-padding for hours, minutes, and seconds under ten.

Источник

Running this produces a clear failure: Falsifying example: test_hours_are_zero_padded(total=5) new_format(5) = '0:0:5' expected hours = '00'

Источник