Build a Two-Tier AI Memory System for Production Agents

Description and use

The article describes moving from in-memory chat history to a two-tier memory system for AI agents. Tier 1 (Redis) stores short-term session context, while Tier 2 (DynamoDB/Postgres) stores long-term user preferences. This reduces token costs, improves reliability, and preserves critical data during container restarts.

Problem

Naive AI agent implementations store chat histories in process memory or dump entire logs into the LLM context window, leading to volatile state loss on restart, high latency and costs due to context bloat, and loss of critical user facts during truncation.

Outcome

The implementation decouples volatile session state from durable user preferences, reducing token costs, improving latency, and ensuring critical user facts are preserved across sessions and container restarts.

Workflow

• Identify the need to decouple ephemeral conversation context from persistent user preferences.
• Implement Tier 1 (L1) using a fast in-memory store or Redis to track active session dialogue turns.
• Implement Tier 2 (L2) using a durable key-value store like DynamoDB, Postgres JSONB, or S3 for structured, long-lived user facts.
• Configure the Agent Controller to read recent context from L1 and structured facts from L2 upon receiving a request.
• Merge the data from L1 and L2 into an optimized prompt context.
• Pass the merged context to the LLM provider instead of the raw full transcript.

Evidence

The naive approach stores chat histories directly in process memory or dumps entire conversation logs back into the LLM context window on every turn.

Source

Implement a two-tier memory architecture: Tier 1 (L1 - Short-Term Session Cache)... Tier 2 (L2 - Long-Term Persistent KV Store).

Source