Review Budget, Not Token Budget: A Pairing Session on Free-Tier Model Access

Описание и применение

Команда ограничила использование бесплатного уровня модели до одного каталога и сопоставила время ревью AI‑патчей с человеческим, чтобы избежать перегрузки.

Задача

Teams often misjudge free-tier AI model limits by focusing on token counts instead of human review capacity, leading to overcommitment and wasted effort.

Результат

The team limited free-tier usage to a single directory and matched AI patch review effort to human capacity, preventing overcommitment and ensuring quality.

Рабочий процесс

• 1. Reframe the question from “which model is best” to “what task to automate”.
• 2. Estimate token usage with a custom script (review_budget.py).
• 3. Run the estimator on the migration branch to get token estimates.
• 4. Measure review effort: count patches and estimate minutes per patch.
• 5. Compare total review minutes to available reviewer capacity.
• 6. Replay three historical patches to validate model reliability.
• 7. Apply a rule: only allow the free tier to touch a single bounded directory.
• 8. Execute the bounded slice, generating patches within review capacity.
• 9. Review AI patches with the same process as human patches.

Доказательства

425,000 tokens against a 10 million allowance. The quota was not the constraint.

Источник

46 draft pull requests. Careful review of each patch took about 15 minutes… 11.5 hours of review attention.

Источник