Building Session-Level Observability for DeepSeek Harness with a Community Plugin

Description and use

The dsh-plugin-agent-insights plugin adds lightweight, session‑scoped observability to DeepSeek Harness without modifying core code. It tracks agent steps, LLM requests, tool calls, durations, failures, and session length, storing data in a WeakMap and cleaning up on session disposal, all while ensuring the agent’s operation is unaffected.

Problem

Agentic systems become hard to debug when a single user request triggers multiple model calls, agent steps, and tool executions, making it unclear how the final response was produced.

Outcome

The plugin provides lightweight, non‑invasive session‑scoped metrics: tool call counts, success/failure, execution duration, slowest tool, LLM request counts, session duration, and cleanup on disposal, all without modifying Harness core.

Workflow

• 1. Register dsh-plugin-agent-insights as a Cordis service (ctx.agentInsights).
• 2. Hook into lifecycle events: agent/request, agent/request-error, tools/execute, session/event, session/disposed.
• 3. For each tool call, wrap execution with performance.now() to record duration, success/failure, and error message.
• 4. Increment counters for total LLM requests and failed requests on agent/request and agent/request-error.
• 5. Store per‑session metrics in a WeakMap<Session, SessionAgentInsights> to avoid memory leaks.
• 6. On session/disposed, retrieve metrics, compute final duration, log a summary, and clean up the WeakMap entry.
• 7. Ensure all metric collection is wrapped in try/catch so observability does not interfere with agent execution.

Evidence

dsh-plugin-agent-insights is a session-scoped observability plugin for DeepSeek Harness.

Source

The plugin wraps the tool execution around next(): const startTime = performance.now() ... const durationMs = performance.now() - startTime

Source

Session completed 8 steps 5 LLM requests 1 failed request 12 tool calls 2 failed tools 14.32s duration Slowest tool: database-query

Source