Provenance gate: no AI agent memory write without a source

Описание и применение

Провенанс‑гейт предотвращает запись в память ИИ без указания источника, блокируя ложные факты и сохраняя аудит.

Задача

AI agents can write false facts into memory if no source is required, leading to unreliable shared knowledge.

Результат

The provenance gate prevented all unsourced, high‑confidence false facts from entering memory, while still allowing legitimate customer‑confirmed updates. The system logged every attempt, enabling auditability.

Рабочий процесс

• 1. Agent proposes a fact.
• 2. Provenance gate checks if the fact has a named source (e.g., a customer utterance).
• 3. If sourced, the fact is written to memory; if not, it is quarantined.
• 4. Quarantined claims are sent to a verifier sub‑agent that defaults to reject.
• 5. Verifier writes the decision and reason to a decision‑journal table in CockroachDB.
• 6. Accepted facts are stored; rejected facts remain in the journal for audit.
• 7. Corrections supersede old facts but keep the old rows marked invalid.

Доказательства

The first version let a whole class of lies through, 6 out of 6 tries. The fixed version blocked 30 out of 30.

Источник

A provenance gate adds one deterministic check before any write commits: does this fact trace to a source?

Источник

The gate still let through 9 of 10 legitimate customer-confirmed updates, not 10 of 10.

Источник

Median gate latency across the run: 293 milliseconds.

Источник