I Built a Terminal AI Assistant That Never Freezes — Here's the Architecture

Description and use

A terminal AI assistant that never freezes, featuring async UI, live Markdown rendering, mid‑conversation model switching, and automatic free‑model list updates from OpenRouter.

Problem

Developers need an AI assistant that can be used directly in the terminal without freezing, allowing seamless switching between free LLM models and preserving conversation context.

Outcome

The terminal AI assistant runs asynchronously, never blocks the UI, renders Markdown live, supports mid‑conversation model switching, and self‑heals the free model list by querying OpenRouter’s catalog and filtering by actual pricing.

Workflow

• 1. Install the CLI via pipx or editable install.
• 2. Run ai-agent to launch the Textual TUI.
• 3. Type prompts; responses render live as Markdown.
• 4. Switch models mid‑conversation using the UI; context is retained.
• 5. The assistant fetches the OpenRouter model catalog on startup, filters free models by pricing, caches for 6 hours, and rebuilds the model list automatically.
• 6. Use keybindings (Enter, Ctrl+L, Ctrl+C, mouse) for interaction.
• 7. Future enhancements include voice engine and persistent memory.

Evidence

The entire top tier had quietly gone paid. The :free suffix in the model id doesn't mean anything once a provider pulls the model from the free tier — OpenRouter just updates the pricing fields in their API.

Source

I wrote openrouter_models.py — a self-healing module that fetches OpenRouter's live model catalog on startup and filters by actual pricing data, not by the id string.

Source