Telling a cheap model to "think a little" made it slower and sometimes silent

说明与用途

一个将请求转发到多个模型系列的网关错误地为 Gemini 2.5 Flash‑Lite 启用了推理功能，而该模型默认不进行推理。这导致在使用紧凑的 token 预算时出现慢速或空响应。通过更新正则表达式排除 Flash‑Lite 并添加测试，确保该模型不接收推理字段，从而解决了问题。

问题

A gateway that forwards requests to multiple model families mistakenly enabled reasoning on a model that does not think by default, causing slow or silent responses when a tight token budget is used.

结果

By refining the regex to exclude Flash‑Lite variants and adding tests, the gateway no longer mistakenly enables reasoning on models that do not think by default, eliminating slow or silent responses under tight token budgets.

工作流

• 1. Build a desktop app that can call a model via Ollama, user API key, or Cloudflare Worker gateway.
• 2. Gateway receives a virtual label and maps it to a real model name.
• 3. Gateway checks if the model name matches a regex for reasoning models.
• 4. If matched, it adds a reasoning field with effort "low" to the request body.
• 5. The model processes the request; if it thinks by default, it may spend all tokens on reasoning and return an empty reply.
• 6. The app observes slower or silent responses when max_tokens is tight.
• 7. Update the regex to exclude "-flash-lite" variants using a negative lookahead.
• 8. Add unit tests to confirm reasoning is not sent to Flash‑Lite and non‑reasoning models are untouched.
• 9. Adjust gateway to treat empty replies as failures rather than successful 200 responses.

证据

The gateway checks the model name and, for models that reason, adds a cap.

来源

The Gemini 2.5 models I had in mind think by default, so gemini-2\.5 looked right.

来源

With a roomier budget the answer can still fit after the thinking, so the failure is quieter: the reply just arrives later than it should, from the model you picked because it was fast.

来源

The fix was one negative lookahead: return /gpt-oss|deepseek-r1|qwq|magistral|gemini-2\.5(?!-flash-lite)/i.test(model);

来源