The Agent Loop Was Fast. The Trace Lied.

说明与用途

将计时拆分为三个独立的跨度（提示构建、模型往返、工具执行），发现主要延迟来自TTFT和工具执行，而非模型本身。

问题

A single timer wrapped the entire agent loop, masking the true source of latency and leading to incorrect optimization decisions.

结果

Separating the timing into three spans exposed that TTFT and tool execution, not the model, were the primary latency sources, enabling targeted optimizations.

工作流

• 1. Wrap each logical phase in a context manager that records start and end times.
• 2. In the prompt_build span, log payload size and other metadata.
• 3. In the model_round_trip span, stream the response, capture TTFT on the first content chunk, and log token usage.
• 4. In the tool_exec span, execute the tool and record success status.
• 5. Append each span as a JSONL line to a trace file for post‑hoc analysis.

证据

You wrap the entire round trip in one clock. The number comes back huge, so you blame the model and start shopping for a smaller one.

来源

You need three spans, not one timer.

来源