5,000 Log Lines, One Free Model Server, 97.4% Success

说明与用途

使用免费LLM的批处理管道以97.4%成功率处理5,000条日志，利用SQLite队列、重试和令牌预算。

问题

Processing 5,000 mixed-format log lines to extract error type, severity, and service using a free model server while handling rate limits, timeouts, and token budget.

结果

The pipeline processed 5,000 logs with 97.4% success, demonstrating that a free model tier can handle batch workloads when combined with retry logic, checkpointing, and token budgeting.

工作流

• Set MC_BASE_URL and MC_API_KEY as environment variables
• Generate 5,000 log lines in three formats
• Create SQLite queue file batch_queue.db
• Read logs, enqueue each line
• For each queued line: build prompt, call OpenAI client with retries (max 4) and exponential backoff (2s base), retry on 429/5xx
• Store successful responses in a results file
• Checkpoint progress in SQLite to resume after crashes

证据

I built one. It survived. Here’s the design.

来源

Survive 429s and timeouts. Resume after a crash. Never exceed the token budget.

来源