Execution Trees, Not More Logs: A Better Debugging Model for AI Agents

Описание и применение

Деревья выполнения позволяют инженерам видеть причинно-следственные связи в работе AI‑агентов, выявлять ошибки, повторные попытки и откаты, а также проверять правила через CI.

Задача

Flat logs of AI agent runs lack causal structure, making it hard to identify which operation caused failures, recoveries, retries, or concurrency issues.

Результат

Execution trees provide a clear, causal view of agent runs, exposing failures, retries, fallbacks, and concurrency, which improves debugging speed and enables deterministic checks in CI.

Рабочий процесс

• 1. Instrument agent code with AgentInspect wrappers (inspectRun, step, step.tool, step.llm).
• 2. Execute the agent run locally, generating a trace directory.
• 3. View the execution tree with npx agent-inspect view <run> --dir .agent-inspect --summary to inspect causal structure.
• 4. Identify suspicious shapes (nested failures, fallbacks, retries, parallelism).
• 5. Encode deterministic checks using agent-inspect check or TraceContract API to enforce invariants (e.g., max retry count).
• 6. Run checks in CI or locally to catch regressions.
• 7. Combine tree inspection with semantic evaluators and human review for content quality.

Доказательства

An execution tree makes the relationship explicit: support-agent ├── plan ├── fetch-inventory (failed: 503) ├── fetch-inventory (success) └── draft-answer

Источник

The tree preserves both facts: the primary path failed; the recovery path completed.

Источник