FAQ: The venv Died. Five State Myths Agents Repeat

Description and use

Probes in local, agent, and CI environments reveal discrepancies, enabling developers to trust only verified actions.

Problem

Agents claim to install packages, set env vars, persist files, run tests, or store secrets, but CI fails due to missing imports or state inconsistencies.

Outcome

Consistent probing across environments exposes state mismatches, enabling developers to trust only verified actions and maintain reliable CI pipelines.

Workflow

• Copy probe scripts to scripts/agent_probes
• Run probes locally and save to probes/local.txt
• Run probes on the agent box and save to probes/box.txt
• Run probes in CI and save to probes/ci.txt
• Diff the three probe outputs and base decisions on the diff

Evidence

Run the same probe in three places. That means local shell, agent box, then CI.

Source

If the three fingerprints differ, I stop trusting the transcript.

Source