Build a Two-Tier AI Memory System for Production Agents

Описание и применение

Статья описывает переход от хранения истории чата в оперативной памяти к двухуровневой системе памяти для AI-агентов. Уровень 1 (Redis) хранит краткосрочный контекст сессии, а Уровень 2 (DynamoDB/Postgres) — долгосрочные предпочтения пользователя. Это снижает затраты на токены, улучшает надежность и сохраняет важные данные при перезапуске контейнеров.

Задача

Naive AI agent implementations store chat histories in process memory or dump entire logs into the LLM context window, leading to volatile state loss on restart, high latency and costs due to context bloat, and loss of critical user facts during truncation.

Результат

The implementation decouples volatile session state from durable user preferences, reducing token costs, improving latency, and ensuring critical user facts are preserved across sessions and container restarts.

Рабочий процесс

• Identify the need to decouple ephemeral conversation context from persistent user preferences.
• Implement Tier 1 (L1) using a fast in-memory store or Redis to track active session dialogue turns.
• Implement Tier 2 (L2) using a durable key-value store like DynamoDB, Postgres JSONB, or S3 for structured, long-lived user facts.
• Configure the Agent Controller to read recent context from L1 and structured facts from L2 upon receiving a request.
• Merge the data from L1 and L2 into an optimized prompt context.
• Pass the merged context to the LLM provider instead of the raw full transcript.

Доказательства

The naive approach stores chat histories directly in process memory or dumps entire conversation logs back into the LLM context window on every turn.

Источник

Implement a two-tier memory architecture: Tier 1 (L1 - Short-Term Session Cache)... Tier 2 (L2 - Long-Term Persistent KV Store).

Источник