Trace First, Blame Later: A Debug Loop for Failing Agent Runs

说明与用途

一种逐步收集日志、工具调用和 diff 的工作流程，用于调试 LLM 代理，快速定位错误。

问题

Agent runs often finish with exit code 0 yet produce incorrect side‑effects, making it hard to determine whether the model or the harness failed.

结果

The workflow provides a clear, repeatable process to capture all signals of an agent run, enabling teams to pinpoint failures and reduce debugging effort.

工作流

• 1. Ensure a git repo, tool‑call stream, and repeatable command exist.
• 2. Run the agent with the trace_bundle.sh script to produce a timestamped bundle in traces/.
• 3. Review the bundle: before.patch, after.patch, agent.log, and tool_calls.jsonl.
• 4. Summarize the tool stream with the provided Python script to identify suspicious calls.
• 5. Apply the debug loop: reproduce, reduce, isolate, fix, repeating until the bundle is clean.

证据

The script below wraps any agent command. It produces a timestamped bundle in traces/.

来源

Four steps. Repeat until the bundle looks clean.

来源