Most of Your Store's AI Bill Is a Classifier in a Chat Costume

Description and use

Many online stores pay for chat models that are actually making decisions, not generating text. By classifying tasks into four rungs and using the appropriate models, costs can be dramatically reduced and performance improved.

Problem

Online stores spend money on chat‑model calls that are actually decision‑making tasks, not text generation, leading to unnecessary costs and latency.

Outcome

Using a calibrated decision model for rung‑three tasks cuts monthly costs from thousands to a few dozen dollars while maintaining decision quality.

Workflow

• Identify the type of decision (rung 1–4).
• Route the task to the correct model: code, rules engine, calibrated classifier, or generative model.
• Collect the model’s output (e.g., boolean, probability, or text).
• Apply business logic based on confidence thresholds or deterministic rules.
• Cache results and limit daily spend to control cost.

Evidence

A model that returns only typed answers - a choice, a score, a yes/no probability - shipped on 15 September at $0.042 per million input tokens with free output.

Source

The rung is set by the shape of the question, not by the tooling you happen to have wired up.

Source

The 80-character limit and the Codice Fiscale checksum are rung one; they should never have reached a model.

Source

The reason and the suggested correction are rung four - both are generated text, and the reason could just as well be a template keyed to which check failed.

Source

At $0.042 per million the decision model costs about $25 a month; Sonnet 5 lands near $1,800 and Fable 5.1 near $9,000.

Source