My local AI agent remembered things I never said. A reader's security review found it.

Описание и применение

CORTEX запоминал факты из текста, который пользователь не предоставлял напрямую, что привело к ошибкам памяти. После внедрения фильтров и строгих тестов проблема устранена, и память теперь хранит только пользовательские данные.

Задача

CORTEX stored user facts from content the user only asked it to process, leading to unintended memory retention.

Результат

All 13 memory cases passed after the fix; unit tests increased from 264 to 318, and memory now holds only user‑provided facts.

Рабочий процесс

• 1. Identify user‑self statements via self_statements() and filter out quoted or fenced text.
• 2. Prevent the model from seeing the filtered material, so it cannot extract facts from unseen JSON.
• 3. Validate each extracted fact with _is_grounded() to ensure it appears in user words.
• 4. Run regression tests marked xfail to confirm the bug is fixed.
• 5. Merge PR and release v1.2.2.
• 6. Provide reset-memory command for users to clear unintended memory.

Доказательства

Mustafa showed it was also storing facts from things I only asked it to process.

Источник

The last one is the nasty one. Remembered facts go into the system prompt, so a note pasted once would have shaped every later session.

Источник

13 memory cases × 3 runs on qwen2.5:7b: 24/39 before, 39/39 after.

Источник

The fix proves itself against his cases, not mine.

Источник