tokeneff: An Open-Source LLM Cost Meter That Runs Locally

Описание и применение

tokeneff — локальный прокси и счетчик токенов для LLM, позволяющий разработчикам видеть расходы в реальном времени, прогнозировать затраты до конца месяца и сравнивать цены между провайдерами, сохраняя данные локально.

Задача

Developers cannot see real‑time LLM token usage and cost until after the fact, making it hard to control spending and adjust usage patterns.

Результат

Developers gain real‑time visibility into token usage and cost, can forecast month‑end spend with confidence, compare pricing across providers and currencies, and keep all data local for privacy.

Рабочий процесс

• Install tokeneff via pip
• Configure provider and API key with tokeneff setup
• Start the local proxy with tokeneff start (listens on localhost:7860)
• Point LLM client base_url to http://localhost:7860/v1
• Use LLM as usual; tokeneff records token usage and cost in real time
• View live meter with tokeneff dashboard or stats with tokeneff stats
• Analyze month‑end forecast and cost comparisons
• Adjust usage or switch to platform mode based on savings

Доказательства

tokeneff runs a tiny local proxy on localhost:7860. You point your LLM client's base_url at it, and it routes your request to the upstream provider using your own API key — BYOK, the key never leaves your machine

Источник

Month-end forecast: based on your usage trend (weighted: last 7 days × 70% + linear extrapolation × 30%), it predicts what you'll spend by month-end — with a confidence score

Источник