LLM Agent Observability: Traces and Sessions, 10 Problems Reproduced

Description and use

Enabling observability of LLM agents via three HTTP headers lets teams diagnose and fix latency, cost, and failure issues, improving performance and cost-efficiency.

Problem

LLM agents lack fine-grained observability, making it hard to diagnose latency, cost, and failure issues.

Outcome

Latency reduced by 51 %, cache hit rate increased to 73 %, and cost per run dropped by 66 % for the support agent example.

Workflow

• Add X-Trace-Id, X-Span-Id, X-Session-Id headers to each request
• Use asyncio.gather to send independent sub-agent checks concurrently
• Remove timestamps from the first line of prompts
• Avoid resending tool results in conversation history

Evidence

Three request headers (X-Trace-Id, X-Span-Id, X-Session-Id) turn the Synthorai gateway's request log into traces and sessions with no tracing SDK.

Source

Sending three independent sub-agent checks together cut their median time from 3.7 s to 1.8 s at the same cost.

Source

Moving a timestamp off a prompt's first line on GPT-6 Luna raised the cache hit rate from 0% to 73%, from an empty cache.

Source