Thousands of alerts, one root cause, and an engine allowed to say "I don't know"

Описание и применение

Antibody – система анализа корневой причины с использованием ИИ, которая агрегирует логи, метрики и трассировки, ранжирует возможные причины и может явно отказаться от ответа, если доказательств недостаточно.

Задача

When a microservice fails, all dependent services fail, producing thousands of alerts that obscure the true root cause.

Результат

Antibody achieved 100% accuracy on clean traffic and 84.5% in hard mode (top‑3 92.9%). When it provides a HIGH confidence label, it is correct 98.5% of the time; LOW labels (abstain) occur 8.3% of the time and reduce wrong guesses to 42.9%.

Рабочий процесс

• Ingest logs, metrics, traces, and service events
• Learn dependency graph from trace spans
• Detect per‑service EWMA baselines, z‑scores, error‑log and error‑span ratios
• Debounce: require 2 bad ticks before counting
• Correlate new abnormal services with open incidents based on call relationships
• Rank root causes using 8 signals (PageRank, error‑log blame, fault signature, recent deploy, past incident match, etc.)
• Assign RCA score (0‑1) and verdict (HIGH/MEDIUM/LOW) via fixed rules
• Suggest fault signature (restart, rollback, scale out)
• Human acknowledges, applies fix, verifies recovery, resolves incident, remembers outcome

Доказательства

When one microservice fails, every service that calls it starts failing too. The on‑call engineer gets thousands of alerts, and the loudest one is usually a victim, not the cause.

Источник

Antibody reads logs, metrics, traces and service events, folds everything that belongs together into one incident, and ranks the probable root causes with evidence and a confidence label.

Источник

The eight ranking signals include personalised PageRank on the anomaly graph, which suspect went wrong first, how many of the callers' error logs blame it, a local fault signature (crash, bad deploy, leak, CPU, slow queries), a recent deploy, and a match with a past resolved incident.

Источник

On the confidence labels in hard mode: HIGH is given 78.6% of the time and is right 98.5% of the time. LOW, the abstain, is given 8.3% of the time, and a forced guess there would have been right only 42.9% of the time.

Источник

The benchmark, including where it loses: 84 scenarios: 7 services, 6 fault types, 2 seeds each, every one with a known cause.

Источник