Most of Your Store's AI Bill Is a Classifier in a Chat Costume
说明与用途
许多在线商店为聊天模型支付费用,但这些模型实际上是在做决策而不是生成文本。通过将任务分类为四个层级并使用合适的模型,可以显著降低成本并提升性能。
问题
Online stores spend money on chat‑model calls that are actually decision‑making tasks, not text generation, leading to unnecessary costs and latency.
结果
Using a calibrated decision model for rung‑three tasks cuts monthly costs from thousands to a few dozen dollars while maintaining decision quality.
工作流
- Identify the type of decision (rung 1–4).
- Route the task to the correct model: code, rules engine, calibrated classifier, or generative model.
- Collect the model’s output (e.g., boolean, probability, or text).
- Apply business logic based on confidence thresholds or deterministic rules.
- Cache results and limit daily spend to control cost.
证据
A model that returns only typed answers - a choice, a score, a yes/no probability - shipped on 15 September at $0.042 per million input tokens with free output.
The rung is set by the shape of the question, not by the tooling you happen to have wired up.
The 80-character limit and the Codice Fiscale checksum are rung one; they should never have reached a model.
The reason and the suggested correction are rung four - both are generated text, and the reason could just as well be a template keyed to which check failed.
At $0.042 per million the decision model costs about $25 a month; Sonnet 5 lands near $1,800 and Fable 5.1 near $9,000.
- 记录日期: 2026-10-01T21:10:49.169Z