How to Detect and Investigate Agent Drift with ZizkaDB

Description and use

The guide outlines a step‑by‑step workflow for detecting and investigating agent drift using ZizkaDB, covering data preparation, baseline creation, metric interpretation, programmatic access to drift signals, and decision making.

Problem

Engineers and team leads need to know quickly whether an agent’s behavior has changed after prompt, model, or tool updates.

Outcome

Implementing this workflow enables teams to detect behavioral drift within days, differentiate noise from real changes, and reduce investigation effort from weeks to hours, yielding significant cost savings.

Workflow

• Make data drift-ready by fixing event types, naming tools, linking events, and using a stable agent name.
• Run the agent long enough to build a robust baseline of several hundred sessions.
• Open the Agent Behavior view, choose an appropriate window (Sessions, 24h, 7d, 30d), and recompute after new data.
• Read the headline drift label, percentage, and score, then examine the health context (sessions analyzed, error rate delta, average session length).
• Inspect the ranked list of biggest changes in event shares and transitions to form a hypothesis.
• Compare baseline and recent panels side‑by‑side to validate the hypothesis against raw numbers.
• Pull the drift signal programmatically via the SDK’s baseline(agent) method for automation.
• Determine if the change was expected by consulting a simple change log and categorizing drift findings.
• Investigate unexpected shifts using Session Replay, db.why(), db.at(), and error rate checks to identify cause categories.
• Decide to Accept, Fix, or Watch the change, and record the outcome in the change log.
• Establish a routine: review drift after every change and weekly otherwise, enforce critical transitions, and use the Fleet feature for multi‑agent monitoring.
• Use the workflow as a release check to catch regressions early and reduce investigation time.

Evidence

Drift compares an agent’s recent behavior with its own established baseline.

Source

Keep event types fixed. Use a small, stable set of types such as user_message, decision, llm_start, llm_end, tool_start, tool_end, assistant_response, and error.

Source

The dashboard offers windows for Sessions, 24h, 7d, and 30d.

Source

Print the full result first and read its fields before you write any logic against it, since I could not confirm the exact structure.

Source