Migration Diary: Building a Token Ledger Before You Trust a Free Model Tier

说明与用途

将 PR 摘要机器人从付费 AI 接口迁移到 MonkeyCode 免费层和免费服务器，加入了令牌跟踪、无状态工作器和回退机制。

问题

A side‑project bot was incurring a $7/month bill on a paid AI endpoint; the team grew, so did the cost. The goal was to eliminate recurring charges by moving inference to a free model tier with a free server, while ensuring reliability and avoiding silent quota exhaustion.

结果

The migration eliminated the $7/month bill, providing a 10M token allowance that comfortably covered the bot’s 140K average daily usage. The stateless worker design ensured reliability on free infrastructure, and the fallback mechanism protected against quota exhaustion.

工作流

• Measure daily token consumption with a log‑parsing script to estimate budget usage.
• Split the bot into a scheduler and a stateless worker that fetches PR diffs fresh each run.
• Deploy the worker to the free server using a minimal shell script and systemd user service.
• Configure the MonkeyCode endpoint via environment variables (BASE_URL, API_KEY, MODEL).
• Implement a wrapper that attempts the free endpoint first and falls back to the paid endpoint on rate‑limit or quota errors, recording fallback usage metrics.

证据

The bot summarized PR descriptions for my team's repository. It cost about $7 per month, which felt harmless until the team doubled and the bill doubled with it.

来源

I parsed my old API call logs and built a tiny script that tallied prompt and completion tokens by day, so the 10M budget had a real number to compare against.

来源

The worker received a PR URL, fetched the diff, called the model, and stored the summary in the repository's existing storage layer.

来源