Most of Your Store's AI Bill Is a Classifier in a Chat Costume
Описание и применение
Многие онлайн‑магазины платят за чат‑модели, которые фактически принимают решения, а не генерируют текст. Классифицируя задачи по четырём ступеням и применяя подходящие модели, можно существенно снизить расходы и улучшить производительность.
Задача
Online stores spend money on chat‑model calls that are actually decision‑making tasks, not text generation, leading to unnecessary costs and latency.
Результат
Using a calibrated decision model for rung‑three tasks cuts monthly costs from thousands to a few dozen dollars while maintaining decision quality.
Рабочий процесс
- Identify the type of decision (rung 1–4).
- Route the task to the correct model: code, rules engine, calibrated classifier, or generative model.
- Collect the model’s output (e.g., boolean, probability, or text).
- Apply business logic based on confidence thresholds or deterministic rules.
- Cache results and limit daily spend to control cost.
Доказательства
A model that returns only typed answers - a choice, a score, a yes/no probability - shipped on 15 September at $0.042 per million input tokens with free output.
The rung is set by the shape of the question, not by the tooling you happen to have wired up.
The 80-character limit and the Codice Fiscale checksum are rung one; they should never have reached a model.
The reason and the suggested correction are rung four - both are generated text, and the reason could just as well be a template keyed to which check failed.
At $0.042 per million the decision model costs about $25 a month; Sonnet 5 lands near $1,800 and Fable 5.1 near $9,000.
- Дата записи: 2026-10-01T21:10:49.169Z