Audit Context Re‑Send Before You Trust a Free Token Allowance

Описание и применение

В многократных взаимодействиях с ИИ переписываемый контекст растёт квадратично, быстро исчерпывая бесплатные лимиты токенов. Методика измерения и корректировки позволяет контролировать расходы.

Задача

In multi‑turn agent loops, the amount of context resent each turn causes token consumption to grow quadratically, making free token allowances unsustainable.

Результат

The workflow reveals a 9.5:1 input‑to‑output ratio in a sample run, highlighting that free token allowances are quickly exhausted if context is not managed.

Рабочий процесс

• Record a real loop with at least one failure, logging each turn as JSONL with prompt, reply, and usage_input if available.
• Estimate context size by counting UTF‑8 bytes and dividing by four to approximate tokens.
• Run the provided audit_loop.py script on the transcript to compute cumulative input, output, and the input‑to‑output ratio.
• Use the measured multiplier to adjust loop design or budget token limits accordingly.

Доказательства

in a multi‑turn agent loop, the transcript you resend — not the model's reply — decides whether a free token allowance survives the day.

Источник

Small teams tend to budget by output length. That is the wrong variable. Every turn ships the whole prior conversation back as input, so consumption grows quadratically while the visible artifact (the answer) grows linearly.

Источник

Turn 1 input: 0 prior tokens. Turn 5 input: ~1,600 prior tokens. Turn 20 input: ~7,600 prior tokens. Cumulative input across 20 turns: ~76,000 tokens. Cumulative output: ~8,000. A 9.5:1 input‑to‑output ratio.

Источник

Record every turn as JSONL with the prompt actually sent and the reply actually received. If the provider returns usage numbers, store them too — they calibrate the estimate later.

Источник