Building Session-Level Observability for DeepSeek Harness with a Community Plugin

Описание и применение

Плагин dsh-plugin-agent-insights добавляет в DeepSeek Harness без изменения ядра подробную, сессий‑ориентированную наблюдаемость: количество шагов, запросов к LLM, вызовов инструментов, их длительность и ошибки, а также длительность сессии. Он использует WeakMap для хранения метрик и очищает их при завершении сессии, не влияя на работу агента.

Задача

Agentic systems become hard to debug when a single user request triggers multiple model calls, agent steps, and tool executions, making it unclear how the final response was produced.

Результат

The plugin provides lightweight, non‑invasive session‑scoped metrics: tool call counts, success/failure, execution duration, slowest tool, LLM request counts, session duration, and cleanup on disposal, all without modifying Harness core.

Рабочий процесс

• 1. Register dsh-plugin-agent-insights as a Cordis service (ctx.agentInsights).
• 2. Hook into lifecycle events: agent/request, agent/request-error, tools/execute, session/event, session/disposed.
• 3. For each tool call, wrap execution with performance.now() to record duration, success/failure, and error message.
• 4. Increment counters for total LLM requests and failed requests on agent/request and agent/request-error.
• 5. Store per‑session metrics in a WeakMap<Session, SessionAgentInsights> to avoid memory leaks.
• 6. On session/disposed, retrieve metrics, compute final duration, log a summary, and clean up the WeakMap entry.
• 7. Ensure all metric collection is wrapped in try/catch so observability does not interfere with agent execution.

Доказательства

dsh-plugin-agent-insights is a session-scoped observability plugin for DeepSeek Harness.

Источник

The plugin wraps the tool execution around next(): const startTime = performance.now() ... const durationMs = performance.now() - startTime

Источник

Session completed 8 steps 5 LLM requests 1 failed request 12 tool calls 2 failed tools 14.32s duration Slowest tool: database-query

Источник