The Formatter Looked Clean. Property Testing Found the Missing Zeroes.

Description and use

An agent patch removed zero‑padding from a formatter, passing unit tests but breaking the contract. A model extracted invariants from the old code, turned them into property tests, and caught the error, blocking the merge.

Problem

An agent patch replaced a hand‑written formatter with a shorter version that silently dropped zero‑padding, causing incorrect output for values under ten, but unit tests did not catch it.

Outcome

The property gate caught a contract violation that unit tests missed, preventing a broken merge and ensuring the formatter’s zero‑padding behavior was preserved.

Workflow

• Check out the old implementation from git.
• Generate a list of invariants from the old code using a language model.
• Convert each invariant into a property test (e.g., zero‑padding for hours, minutes, seconds).
• Run the properties against both old and new implementations using Hypothesis.
• Block the merge if any property holds on the old version but fails on the new one.

Evidence

The new formatter silently dropped zero-padding for hours, minutes, and seconds under ten.

Source

Running this produces a clear failure: Falsifying example: test_hours_are_zero_padded(total=5) new_format(5) = '0:0:5' expected hours = '00'

Source