The Agent Log Is a Claim: Five Trace Myths

Описание и применение

Проверка фактического выполнения команд агентом с помощью скрипта snapshot‑audit гарантирует, что утверждения в логе соответствуют реальному состоянию репозитория и контрактам.

Задача

Agents may claim to run commands or tests, but logs can be misleading; verification of actual execution is required to avoid merge failures.

Результат

By auditing repository state and command execution, developers avoid false positives from agent logs and ensure CI passes before merging.

Рабочий процесс

• 1. Capture a snapshot of the repository tree before the agent session.
• 2. Run the agent and record its tool calls, working directories, and exit codes.
• 3. Capture a second snapshot after the session.
• 4. Diff the two snapshots and compute file hashes to verify claimed edits.
• 5. Compare exit codes and logs against expected contracts.
• 6. If discrepancies exist, replay the command locally to confirm execution.
• 7. Merge only when all artifacts satisfy the contracts.

Доказательства

An agent prints pytest and a green block. I almost merge. Then CI screams.

Источник

The script does not talk to a model. It snapshots a tree, diffs it, then runs your contract.

Источник