Free Tokens, Free Server, and a Property Test That Refuses to Be Gamed

Description and use

LLM‑generated patches often pass unit tests but still corrupt global state. By running a property‑based harness on a free server with many random seeds, developers can quickly catch subtle bugs that unit tests miss.

Problem

LLM‑generated patches often pass unit tests but still corrupt global state, leading to failures in production.

Outcome

The property harness detected a subtle bug in a vector‑based ring buffer that unit tests missed, proving that randomised property tests on a free server are an effective, low‑cost verification strategy.

Workflow

• Ask MonkeyCode’s free model to generate a patch
• Push the patch to a free server with a property‑test harness
• Run the harness with many random seeds and steps
• If the harness fails, review and fix the patch
• Repeat until the harness passes all seeds

Evidence

A property test with 1000 random seeds would have caught it at seed 17.

Source

The first time I ran this against an agent-generated ring buffer, the failure appeared at seed 17.

Source