5,000 Log Lines, One Free Model Server, 97.4% Success

Description and use

A batch pipeline processes 5,000 logs with 97.4% success using a free LLM, leveraging SQLite queue, retries, and token budgeting.

Problem

Processing 5,000 mixed-format log lines to extract error type, severity, and service using a free model server while handling rate limits, timeouts, and token budget.

Outcome

The pipeline processed 5,000 logs with 97.4% success, demonstrating that a free model tier can handle batch workloads when combined with retry logic, checkpointing, and token budgeting.

Workflow

• Set MC_BASE_URL and MC_API_KEY as environment variables
• Generate 5,000 log lines in three formats
• Create SQLite queue file batch_queue.db
• Read logs, enqueue each line
• For each queued line: build prompt, call OpenAI client with retries (max 4) and exponential backoff (2s base), retry on 429/5xx
• Store successful responses in a results file
• Checkpoint progress in SQLite to resume after crashes

Evidence

I built one. It survived. Here’s the design.

Source

Survive 429s and timeouts. Resume after a crash. Never exceed the token budget.

Source