Cap LLM Spend per Team with Virtual Keys

说明与用途

在网关层实现了LLM请求成本跟踪、归属和预算执行。

问题

Organizations need to know who is spending on LLMs and prevent runaway costs.

结果

Implemented per‑request cost tracking, attribution, and budget enforcement at the gateway level, enabling real‑time spend visibility and automated request blocking.

工作流

• Load a JSON price catalog into a ConfigMap and reference it via EnterpriseAgentgatewayParameters.
• Create virtual keys as Secrets containing user/team metadata.
• Configure the gateway to use Strict API key authentication and enable entBudgetEnforcement.
• Define budgets in an EnterpriseAgentgatewayBudget resource with token and USD limits.
• Run the demo with ./demo.sh --act 8 to observe cost attribution and budget enforcement.
• Verify budgets via kubectl get eagbud and monitor logs for llm.cost fields.

证据

The gateway picks it up through modelCatalog.sources on the EnterpriseAgentgatewayParameters it already references

来源

A virtual key is a gateway‑issued token carrying attribution metadata

来源

Strict means a valid key is required, so on this route unattributed spend cannot happen at all rather than merely being visible

来源