Build a Tiny Token Meter Before You Trust a Free Model Tier

Description and use

A lightweight HTTP proxy that records token usage per request to a free AI endpoint, preventing accidental quota exhaustion and logging usage in JSONL.

Problem

Free model tiers can be exhausted silently by retry loops that ignore usage data, leading to unexpected quota depletion before a demo or usage dashboard is checked.

Outcome

The token meter prevented accidental quota exhaustion, provided per‑request token accounting, and logged usage for auditability.

Workflow

• Configure environment variables: UPSTREAM_URL, UPSTREAM_TOKEN, DAY_TOKEN_CAP, LEDGER_PATH, PORT, METER_STUB.
• Run meter.py as a local HTTP proxy on the specified PORT.
• Client sends requests to the proxy instead of directly to the upstream endpoint.
• Proxy forwards the request, receives the response, extracts prompt and completion tokens from the usage block.
• Proxy writes a JSON line to the ledger with day, timestamp, prompt_tokens, completion_tokens, and total_tokens.
• Proxy checks the accumulated tokens for the day; if the sum exceeds DAY_TOKEN_CAP, it returns an error and refuses new calls.
• Optional stub mode (METER_STUB=1) returns a deterministic response for testing without consuming real tokens.

Evidence

The meter should do three things and nothing else. Count tokens per request from the upstream's own usage block, append one JSON line per call to a ledger, and refuse new calls once today's total crosses a hard cap.

Source

One file, meter.py. It is an HTTP proxy: you point your client at it, it forwards to the upstream, it records what came back.

Source

The upstream call has a stub branch, and that branch is the whole reason this article is testable at 11:40pm without spending anything.

Source