Execution Trees, Not More Logs: A Better Debugging Model for AI Agents

Описание и применение

Деревья выполнения позволяют разработчикам видеть структуру работы AI‑агентов, выявлять причины ошибок, повторные попытки и параллельные операции, а также быстро создавать проверочные правила.

Задача

Flat logs of AI agent runs lack causal structure, making it hard to identify which operation caused failures, recoveries, retries, or concurrency issues.

Результат

Using execution trees, developers can quickly pinpoint where failures occur, understand recovery behavior, detect unnecessary retries, and verify concurrency, leading to more reliable AI agent behavior.

Рабочий процесс

• 1. Instrument agent code with AgentInspect wrappers (inspectRun, step, step.tool, step.llm).
• 2. Execute the agent run locally, generating a trace in .agent-inspect directory.
• 3. View the execution tree with npx agent-inspect view <run> --dir .agent-inspect --summary to inspect the causal structure.
• 4. Identify suspicious patterns (e.g., retries, fallbacks, parallelism).
• 5. Encode deterministic checks using agent-inspect check or TraceContract API to enforce invariants (e.g., max retry count, required tools).
• 6. Run checks in CI to catch regressions.
• 7. Combine tree inspection with semantic evaluators and human review for content quality.

Доказательства

An execution tree makes the relationship explicit: support-agent ├── plan ├── fetch-inventory (failed: 503) ├── fetch-inventory (success) └── draft-answer

Источник

The tree preserves both facts: the primary path failed; the recovery path completed.

Источник

npx agent-inspect check travel-planner --dir .agent-inspect --preset trajectory --required-tool search-flights --fail-on-observation failed

Источник