tokeneff: An Open-Source LLM Cost Meter That Runs Locally

说明与用途

tokeneff 是一个本地代理和令牌计数器，帮助 LLM 开发者实时查看成本、预测月底支出并比较不同供应商价格，同时保持所有数据本地化。

问题

Developers cannot see real‑time LLM token usage and cost until after the fact, making it hard to control spending and adjust usage patterns.

结果

Developers gain real‑time visibility into token usage and cost, can forecast month‑end spend with confidence, compare pricing across providers and currencies, and keep all data local for privacy.

工作流

• Install tokeneff via pip
• Configure provider and API key with tokeneff setup
• Start the local proxy with tokeneff start (listens on localhost:7860)
• Point LLM client base_url to http://localhost:7860/v1
• Use LLM as usual; tokeneff records token usage and cost in real time
• View live meter with tokeneff dashboard or stats with tokeneff stats
• Analyze month‑end forecast and cost comparisons
• Adjust usage or switch to platform mode based on savings

证据

tokeneff runs a tiny local proxy on localhost:7860. You point your LLM client's base_url at it, and it routes your request to the upstream provider using your own API key — BYOK, the key never leaves your machine

来源

Month-end forecast: based on your usage trend (weighted: last 7 days × 70% + linear extrapolation × 30%), it predicts what you'll spend by month-end — with a confidence score

来源