Debugging AI Apps Shouldn't Mean Grepping Five Dashboards — Introducing Obyflow

Описание и применение

Obyflow объединяет логи, метрики и трассировки для приложений с LLM и векторными базами, автоматически связывая события и выдавая объяснения с помощью LLM.

Задача

Traditional observability tools miss failure modes of AI-heavy apps, making it hard to pinpoint issues like empty LLM results or slow vector queries.

Результат

Obyflow reduces debugging time by consolidating evidence, auto‑correlating traces, and providing LLM‑driven explanations with confidence scores.

Рабочий процесс

• Install Obyflow CLI and SDKs
• Add Obyflow instrumentation to your service (e.g., Node.js start() or Python middleware)
• Instrument vector stores and LLM calls via SDK adapters
• Run "obyflow investigate" to gather evidence for a time window
• Ask the LLM "why did checkout fail today?" to get a grounded explanation
• Review confidence labels and grounding validation
• Iterate on instrumentation or configuration as needed

Доказательства

Obyflow captures structured events locally — using SQLite, with no external backend required — correlates them into full traces, and uses an LLM of your choice to turn the raw evidence into a plain-English root-cause investigation.

Источник

The investigation also goes through grounding validation, which flags parts of the answer that don't match the underlying evidence.

Источник