Bringing Live Application Logs to Reel Quick

Description and use

Reel Quick implemented a centralized real-time log viewer. Logs from FastAPI and workers are written to Redis Streams and streamed to the frontend via SSE. This simplifies debugging of the distributed system.

Problem

Debugging a distributed system with FastAPI, asynchronous workers, Redis, and Next.js requires checking multiple terminals and service logs, making it difficult to correlate events.

Outcome

Implemented a centralized, real-time log viewer that aggregates structured logs from backend and workers via Redis Streams and SSE, improving debugging efficiency and operational visibility.

Workflow

• FastAPI backend and workers write structured Python log events to the Redis Stream 'reelquick:system:logs'.
• Each event includes timestamp, severity, service, source, message, and context (e.g., video/job ID).
• The Next.js server proxies protected FastAPI log endpoints to the browser, keeping Redis credentials server-side.
• The frontend loads recent history from Redis Streams and maintains live updates via Server-Sent Events (SSE).
• Users filter logs by severity (debug, info, warning, error, critical) and service (backend, workers).
• Users expand exception details to inspect failure contexts.

Evidence

Reel Quick now writes structured Python log events to the Redis Stream reelquick:system:logs.

Source

The Next.js server proxies the protected FastAPI log endpoints, which means the browser never receives Redis credentials or the logging API key.

Source