Audit Context Re‑Send Before You Trust a Free Token Allowance

Description and use

In multi‑turn AI interactions, resent context grows quadratically, quickly exhausting free token limits. Measuring and adjusting the re‑send multiplier lets teams manage token usage.

Problem

In multi‑turn agent loops, the amount of context resent each turn causes token consumption to grow quadratically, making free token allowances unsustainable.

Outcome

The workflow reveals a 9.5:1 input‑to‑output ratio in a sample run, highlighting that free token allowances are quickly exhausted if context is not managed.

Workflow

• Record a real loop with at least one failure, logging each turn as JSONL with prompt, reply, and usage_input if available.
• Estimate context size by counting UTF‑8 bytes and dividing by four to approximate tokens.
• Run the provided audit_loop.py script on the transcript to compute cumulative input, output, and the input‑to‑output ratio.
• Use the measured multiplier to adjust loop design or budget token limits accordingly.

Evidence

in a multi‑turn agent loop, the transcript you resend — not the model's reply — decides whether a free token allowance survives the day.

Source

Small teams tend to budget by output length. That is the wrong variable. Every turn ships the whole prior conversation back as input, so consumption grows quadratically while the visible artifact (the answer) grows linearly.

Source

Turn 1 input: 0 prior tokens. Turn 5 input: ~1,600 prior tokens. Turn 20 input: ~7,600 prior tokens. Cumulative input across 20 turns: ~76,000 tokens. Cumulative output: ~8,000. A 9.5:1 input‑to‑output ratio.

Source

Record every turn as JSONL with the prompt actually sent and the reply actually received. If the provider returns usage numbers, store them too — they calibrate the estimate later.

Source