Stop AI Agent Memory Poisoning at the Write Path

Описание и применение

Защита памяти агента от «memory poisoning» достигается путем внедрения шлюза, проверяющего каждую запись перед сохранением. Это предотвращает сохранение вредоносных инструкций и сохраняет целостность решений агента.

Задача

AI agents can be poisoned by malicious prompts that are stored as facts, causing future sessions to act on false instructions and compromise decisions.

Результат

The write‑gate prevents memory poisoning, limiting a single poisoned fact to affect only one key‑value lookup and all four booking decisions in the graph backend, thereby preserving agent integrity across sessions.

Рабочий процесс

• 1. Wrap the MemoryStore with GatedMemoryStore, providing regex rules and an optional LLM classifier.
• 2. In GatedMemoryStore.add(), screen content; if not allowed, raise MemoryRejected.
• 3. If an LLM classifier is configured, invoke it; if unsafe, raise MemoryRejected.
• 4. On successful screening, forward the write to the inner store.
• 5. Configure the Agent’s MemoryManager to use the gated store.
• 6. When a write is rejected, the agent’s add_memory tool returns a failure, and the agent informs the user.
• 7. Deploy the agent behind any entry point (chat, API, Lambda) so the gate travels with the agent.

Доказательства

one poisoned fact skews 1 lookup in key-value memory, but hijacks 4/4 booking decisions in a graph

Источник

A rejected write raises rather than dropping silently

Источник