Audit Context Re‑Send Before You Trust a Free Token Allowance

说明与用途

在多轮 AI 交互中，重发的上下文呈二次增长，迅速耗尽免费令牌额度。通过测量并调整重发倍数，团队可控制令牌使用量。

问题

In multi‑turn agent loops, the amount of context resent each turn causes token consumption to grow quadratically, making free token allowances unsustainable.

结果

The workflow reveals a 9.5:1 input‑to‑output ratio in a sample run, highlighting that free token allowances are quickly exhausted if context is not managed.

工作流

• Record a real loop with at least one failure, logging each turn as JSONL with prompt, reply, and usage_input if available.
• Estimate context size by counting UTF‑8 bytes and dividing by four to approximate tokens.
• Run the provided audit_loop.py script on the transcript to compute cumulative input, output, and the input‑to‑output ratio.
• Use the measured multiplier to adjust loop design or budget token limits accordingly.

证据

in a multi‑turn agent loop, the transcript you resend — not the model's reply — decides whether a free token allowance survives the day.

来源

Small teams tend to budget by output length. That is the wrong variable. Every turn ships the whole prior conversation back as input, so consumption grows quadratically while the visible artifact (the answer) grows linearly.

来源

Turn 1 input: 0 prior tokens. Turn 5 input: ~1,600 prior tokens. Turn 20 input: ~7,600 prior tokens. Cumulative input across 20 turns: ~76,000 tokens. Cumulative output: ~8,000. A 9.5:1 input‑to‑output ratio.

来源

Record every turn as JSONL with the prompt actually sent and the reply actually received. If the provider returns usage numbers, store them too — they calibrate the estimate later.

来源