Weir – deterministic unit tests for AI agents

Описание и применение

Weir обеспечивает детерминированные, без LLM, юнит‑тесты для ИИ‑агентов, автоматически обнаруживая запрещённые потоки данных и прерывая CI с подробным путём доказательства.

Задача

AI agents can silently misbehave, sending sensitive data to forbidden sinks while logs claim success, making it hard to detect failures without relying on another LLM.

Результат

Weir provides deterministic, LLM‑free unit tests for AI agents, automatically detecting forbidden data flows and failing CI with a clear witness path.

Рабочий процесс

• 1. Instrument agent to emit OpenTelemetry traces.
• 2. Run weir gauge to check trace coverage and instrumentation completeness.
• 3. If coverage is low, adjust instrumentation and re‑run.
• 4. Run weir scan on the exported traces.
• 5. Weir reconstructs the session graph and tracks taint through it.
• 6. If a forbidden flow is detected, Weir exits with code 1 and outputs a witness path.
• 7. CI fails the build, alerting developers to the issue.

Доказательства

It reads the OpenTelemetry traces your agent already emits and fails the build when sensitive data reaches a sink it should not reach.

Источник

Same input, byte‑identical answer, no LLM in the loop, nothing leaves your machine.

Источник