48-Hour Field Notes: I Built a Token Ledger Before Trusting Any Free Tier

Description and use

A local token ledger and summary script were built to accurately account for MonkeyCode's free tier usage, with added retry logging and a smoke test to detect server failures.

Problem

Determine accurate token usage and failure handling for a free-tier AI service before deploying an agent loop.

Outcome

Accurate token accounting and failure detection enabled safe use of the free tier; retries and server changes were identified and mitigated.

Workflow

• 1. Implement meter.py to append a row per model call with provider, model, prompt_tokens, completion_tokens, total_tokens, and note.
• 2. Wrap each API call to log an "attempt" row before sending and an "outcome" row after receiving a response or error.
• 3. Run the loop with retry on 429 and read timeouts, ensuring all attempts are logged regardless of success.
• 4. Execute summary.py to compute calls, p50 tokens, null counts, span, and burn rate from the ledger.
• 5. Perform a smoke test using curl against the current base URL and model ID to check for truncation and latency issues on the free server.

Evidence

"The ledger lives locally, and it records what each call reported about itself."

Source

"I ran a loop that retried on 429 and on read timeouts, and the ledger showed 41 calls while the client had issued 58."

Source

"Moving a worker onto a remote free server felt like a speed decision, and it was really a lifecycle decision."

Source