How I track per-customer LLM costs in production

说明与用途

作者描述了在多租户应用程序中按客户跟踪 LLM 成本的解决方案。与其依赖提供商的汇总发票，不如在请求元数据中传递 tenant_id，并使用 Next.js、Supabase 和 Inngest 构建具有预算超额警报的实时管道。介绍了一个开源工具 LLMeter。

问题

Standard billing dashboards show total LLM spend but fail to attribute costs to specific customers or tenants, making it difficult to monitor margins and prevent budget overruns by power users.

结果

The implementation allows for real-time tracking of LLM costs per tenant, enabling proactive budget alerts and accurate margin analysis without over-engineering the billing stack.

工作流

• Pass user_id or tenant_id as metadata in every LLM API request (e.g., OpenAI 'user' field, Anthropic custom headers).
• Proxy requests or capture token usage post-request to extract cost data.
• Fire an event to an async worker (Inngest) containing tenant_id, model, and token counts.
• Write usage data to a time-series database (Supabase).
• Trigger alerts when cumulative costs exceed defined budget limits for a tenant.
• Visualize data using a dashboard (LLMeter) for per-model, per-user, and per-day tracking.

证据

Pass the user_id or tenant_id directly to the provider... This simple line changes everything.

来源

We ended up building a pipeline for this: Next.js edge functions for ingestion, Supabase for fast time-series queries, and Inngest for async budget checking.

来源