LLM Agent Observability: Traces and Sessions, 10 Problems Reproduced

Описание и применение

Обеспечение наблюдаемости LLM‑агентов через три HTTP‑заголовка позволяет выявлять и устранять проблемы с задержкой, стоимостью и отказами, улучшая производительность и экономичность.

Задача

LLM agents lack fine-grained observability, making it hard to diagnose latency, cost, and failure issues.

Результат

Latency reduced by 51 %, cache hit rate increased to 73 %, and cost per run dropped by 66 % for the support agent example.

Рабочий процесс

• Add X-Trace-Id, X-Span-Id, X-Session-Id headers to each request
• Use asyncio.gather to send independent sub-agent checks concurrently
• Remove timestamps from the first line of prompts
• Avoid resending tool results in conversation history

Доказательства

Three request headers (X-Trace-Id, X-Span-Id, X-Session-Id) turn the Synthorai gateway's request log into traces and sessions with no tracing SDK.

Источник

Sending three independent sub-agent checks together cut their median time from 3.7 s to 1.8 s at the same cost.

Источник

Moving a timestamp off a prompt's first line on GPT-6 Luna raised the cache hit rate from 0% to 73%, from an empty cache.

Источник