Quarantine Agent Patches on a Free Ephemeral Server

Description and use

Validating AI‑generated patches on a clean, disposable server uncovers hidden bugs that local tests miss, reducing the risk of shipping faulty code.

AI‑generated patches often pass local tests but fail in a different environment due to cached state or hidden assumptions.

The quarantine workflow catches hidden bugs (e.g., timezone issues) that local tests miss, reducing the risk of shipping faulty patches.

Create a branch with the agent’s patch and open a PR or equivalent
Start a free MonkeyCode server and obtain a shell
Clone the repository on the server using a shallow clone (git clone --depth 1)
Check out the patch branch
Install the project in editable mode (pip install -e .)
Run focused tests (e.g., property‑based tests for the patched files) with pytest -q
If a test fails, analyze the traceback and optionally ask an LLM for hypotheses
Use fixtures to pin environment variables (e.g., timezone) to avoid state leakage
Mark flaky tests with @pytest.mark.skipif and run them manually when ready
Wrap the steps in a quarantine.sh script that SSHs into the server and executes the commands
Repeat for each suspicious patch

I now do it for every suspicious one. It caught two real bugs in the last week. Both were timezone issues.