Thousands of alerts, one root cause, and an engine allowed to say "I don't know"

Description and use

Antibody is an AI‑powered root‑cause analysis engine that aggregates logs, metrics, traces, and events, ranks probable causes, and can explicitly abstain when evidence is insufficient.

Problem

When a microservice fails, all dependent services fail, producing thousands of alerts that obscure the true root cause.

Outcome

Antibody achieved 100% accuracy on clean traffic and 84.5% in hard mode (top‑3 92.9%). When it provides a HIGH confidence label, it is correct 98.5% of the time; LOW labels (abstain) occur 8.3% of the time and reduce wrong guesses to 42.9%.

Workflow

• Ingest logs, metrics, traces, and service events
• Learn dependency graph from trace spans
• Detect per‑service EWMA baselines, z‑scores, error‑log and error‑span ratios
• Debounce: require 2 bad ticks before counting
• Correlate new abnormal services with open incidents based on call relationships
• Rank root causes using 8 signals (PageRank, error‑log blame, fault signature, recent deploy, past incident match, etc.)
• Assign RCA score (0‑1) and verdict (HIGH/MEDIUM/LOW) via fixed rules
• Suggest fault signature (restart, rollback, scale out)
• Human acknowledges, applies fix, verifies recovery, resolves incident, remembers outcome

Evidence

When one microservice fails, every service that calls it starts failing too. The on‑call engineer gets thousands of alerts, and the loudest one is usually a victim, not the cause.

Source

Antibody reads logs, metrics, traces and service events, folds everything that belongs together into one incident, and ranks the probable root causes with evidence and a confidence label.

Source

The eight ranking signals include personalised PageRank on the anomaly graph, which suspect went wrong first, how many of the callers' error logs blame it, a local fault signature (crash, bad deploy, leak, CPU, slow queries), a recent deploy, and a match with a past resolved incident.

Source

On the confidence labels in hard mode: HIGH is given 78.6% of the time and is right 98.5% of the time. LOW, the abstain, is given 8.3% of the time, and a forced guess there would have been right only 42.9% of the time.

Source

The benchmark, including where it loses: 84 scenarios: 7 services, 6 fault types, 2 seeds each, every one with a known cause.

Source