When to fine‑tune a small model for one narrow task

说明与用途

Intercom 将 GPT‑4.1 替换为微调后的 Qwen 模型，用于规范化任务，将推理成本从约 25 万美元/月降至几乎为零。

问题

High inference cost on a narrow, high‑volume ancillary task using a frontier LLM.

结果

Intercom reduced inference spend on the summarization task from an estimated $250k/month to almost zero by swapping GPT‑4.1 for a fine‑tuned Qwen model.

工作流

• 1. Identify ancillary, repeatable prompt that is close to a well‑prompted but untrained open model. 2. Run a production test with the untrained open model to confirm performance is acceptable. 3. Fine‑tune a small open‑weight model (e.g., Qwen 3) on the task. 4. Deploy the fine‑tuned model in place of the frontier model for that step. 5. Monitor resolution metrics and cost savings in production.

证据

I don't remember exactly, but it could have been like a quarter million dollars a month on just that summarization task.

来源

saved almost all of those, you know, several hundred thousand a month on inference just by doing that one thing.

来源