Execution Trees, Not More Logs: A Better Debugging Model for AI Agents

说明与用途

执行树帮助开发者可视化 AI 代理的执行路径，识别错误原因、重试和并行操作，并快速创建确定性检查。

问题

Flat logs of AI agent runs lack causal structure, making it hard to identify which operation caused failures, recoveries, retries, or concurrency issues.

结果

Using execution trees, developers can quickly pinpoint where failures occur, understand recovery behavior, detect unnecessary retries, and verify concurrency, leading to more reliable AI agent behavior.

工作流

• 1. Instrument agent code with AgentInspect wrappers (inspectRun, step, step.tool, step.llm).
• 2. Execute the agent run locally, generating a trace in .agent-inspect directory.
• 3. View the execution tree with npx agent-inspect view <run> --dir .agent-inspect --summary to inspect the causal structure.
• 4. Identify suspicious patterns (e.g., retries, fallbacks, parallelism).
• 5. Encode deterministic checks using agent-inspect check or TraceContract API to enforce invariants (e.g., max retry count, required tools).
• 6. Run checks in CI to catch regressions.
• 7. Combine tree inspection with semantic evaluators and human review for content quality.

证据

An execution tree makes the relationship explicit: support-agent ├── plan ├── fetch-inventory (failed: 503) ├── fetch-inventory (success) └── draft-answer

来源

The tree preserves both facts: the primary path failed; the recovery path completed.

来源

npx agent-inspect check travel-planner --dir .agent-inspect --preset trajectory --required-tool search-flights --fail-on-observation failed

来源