The Agent Log Is a Claim: Five Trace Myths

Description and use

Verifying an agent’s claimed execution with a snapshot‑audit script ensures log claims match the actual repository state and contracts.

Agents may claim to run commands or tests, but logs can be misleading; verification of actual execution is required to avoid merge failures.

By auditing repository state and command execution, developers avoid false positives from agent logs and ensure CI passes before merging.

1. Capture a snapshot of the repository tree before the agent session.
2. Run the agent and record its tool calls, working directories, and exit codes.
3. Capture a second snapshot after the session.
4. Diff the two snapshots and compute file hashes to verify claimed edits.
5. Compare exit codes and logs against expected contracts.
6. If discrepancies exist, replay the command locally to confirm execution.
7. Merge only when all artifacts satisfy the contracts.

An agent prints pytest and a green block. I almost merge. Then CI screams.

The script does not talk to a model. It snapshots a tree, diffs it, then runs your contract.