Tokens Are Not the Metric

Description and use

<p>When developers start using AI seriously, one metric appears almost immediately: <strong>token usage</strong>.</p> <ul> <li>How many tokens did this agent consume?</li> <li>How much did this coding session cost?</li> <li>Can we reduce the context window?</li> <li>Can we use a cheaper model?</li> </ul> <p>Those are reasonable questions. But they can also lead us toward the wrong optimization.</p> <blockquote> <p>The goal should not be to use fewer tokens.<br> The goal should be to <strong>waste</strong> fewer tokens.</p> </blockquote> <p>There is a big difference.</p> <h2> A million tokens c

Problem

<p>When developers start using AI seriously, one metric appears almost immediately: <strong>token usage</strong>.</p> <ul> <li>How many tokens did this agent consume?</li> <li>How much did this coding session cost?</li> <li>Can we reduce the context window?</li> <li>Can we use a cheaper model?</li> </ul> <p>Those are reasonable questions. But they can also lead us toward the wrong optimization.</p> <blockquote> <p>The goal should not be to use fewer tokens.<br> The goal should be to <strong>waste</strong> fewer tokens.</p> </blockquote> <p>There is a big difference.</p> <h2> A million tokens c

Outcome

<p>When developers start using AI seriously, one metric appears almost immediately: <strong>token usage</strong>.</p> <ul> <li>How many tokens did this agent consume?</li> <li>How much did this coding session cost?</li> <li>Can we reduce the context window?</li> <li>Can we use a cheaper model?</li> </ul> <p>Those are reasonable questions. But they can also lead us toward the wrong optimization.</p> <blockquote> <p>The goal should not be to use fewer tokens.<br> The goal should be to <strong>waste</strong> fewer tokens.</p> </blockquote> <p>There is a big difference.</p> <h2> A million tokens c

Evidence

Tokens Are Not the Metric

Source