tokeneff: An Open-Source LLM Cost Meter That Runs Locally

Description and use

tokeneff is a local proxy and token meter for LLMs that gives developers real‑time cost visibility, month‑end forecasting, and cross‑provider pricing comparison while keeping all data local.

Problem

Developers cannot see real‑time LLM token usage and cost until after the fact, making it hard to control spending and adjust usage patterns.

Outcome

Developers gain real‑time visibility into token usage and cost, can forecast month‑end spend with confidence, compare pricing across providers and currencies, and keep all data local for privacy.

Workflow

• Install tokeneff via pip
• Configure provider and API key with tokeneff setup
• Start the local proxy with tokeneff start (listens on localhost:7860)
• Point LLM client base_url to http://localhost:7860/v1
• Use LLM as usual; tokeneff records token usage and cost in real time
• View live meter with tokeneff dashboard or stats with tokeneff stats
• Analyze month‑end forecast and cost comparisons
• Adjust usage or switch to platform mode based on savings

Evidence

tokeneff runs a tiny local proxy on localhost:7860. You point your LLM client's base_url at it, and it routes your request to the upstream provider using your own API key — BYOK, the key never leaves your machine

Source

Month-end forecast: based on your usage trend (weighted: last 7 days × 70% + linear extrapolation × 30%), it predicts what you'll spend by month-end — with a confidence score

Source