Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

How to Detect and Investigate Agent Drift with ZizkaDB

Описание и применение

Руководство описывает пошаговый процесс обнаружения и расследования изменения поведения агента с помощью ZizkaDB, включая подготовку данных, построение базового уровня, интерпретацию метрик, программный доступ к сигналу дрейфа и принятие решений о корректировке.

Задача

Engineers and team leads need to know quickly whether an agent’s behavior has changed after prompt, model, or tool updates.

Результат

Implementing this workflow enables teams to detect behavioral drift within days, differentiate noise from real changes, and reduce investigation effort from weeks to hours, yielding significant cost savings.

Рабочий процесс

  • Make data drift-ready by fixing event types, naming tools, linking events, and using a stable agent name.
  • Run the agent long enough to build a robust baseline of several hundred sessions.
  • Open the Agent Behavior view, choose an appropriate window (Sessions, 24h, 7d, 30d), and recompute after new data.
  • Read the headline drift label, percentage, and score, then examine the health context (sessions analyzed, error rate delta, average session length).
  • Inspect the ranked list of biggest changes in event shares and transitions to form a hypothesis.
  • Compare baseline and recent panels side‑by‑side to validate the hypothesis against raw numbers.
  • Pull the drift signal programmatically via the SDK’s baseline(agent) method for automation.
  • Determine if the change was expected by consulting a simple change log and categorizing drift findings.
  • Investigate unexpected shifts using Session Replay, db.why(), db.at(), and error rate checks to identify cause categories.
  • Decide to Accept, Fix, or Watch the change, and record the outcome in the change log.
  • Establish a routine: review drift after every change and weekly otherwise, enforce critical transitions, and use the Fleet feature for multi‑agent monitoring.
  • Use the workflow as a release check to catch regressions early and reduce investigation time.

Доказательства

Drift compares an agent’s recent behavior with its own established baseline.

Источник

Keep event types fixed. Use a small, stable set of types such as user_message, decision, llm_start, llm_end, tool_start, tool_end, assistant_response, and error.

Источник

The dashboard offers windows for Sessions, 24h, 7d, and 30d.

Источник

Print the full result first and read its fields before you write any logic against it, since I could not confirm the exact structure.

Источник

  • Дата записи: 2026-10-23T11:11:15.024Z