Telling a cheap model to "think a little" made it slower and sometimes silent

Description and use

A gateway that forwards requests to multiple model families mistakenly enabled reasoning on Gemini 2.5 Flash‑Lite, which does not think by default. This caused slow or silent responses when a tight token budget was used. The fix updated the regex to exclude Flash‑Lite and added tests to ensure the reasoning field is omitted for that model.

Problem

A gateway that forwards requests to multiple model families mistakenly enabled reasoning on a model that does not think by default, causing slow or silent responses when a tight token budget is used.

Outcome

By refining the regex to exclude Flash‑Lite variants and adding tests, the gateway no longer mistakenly enables reasoning on models that do not think by default, eliminating slow or silent responses under tight token budgets.

Workflow

• 1. Build a desktop app that can call a model via Ollama, user API key, or Cloudflare Worker gateway.
• 2. Gateway receives a virtual label and maps it to a real model name.
• 3. Gateway checks if the model name matches a regex for reasoning models.
• 4. If matched, it adds a reasoning field with effort "low" to the request body.
• 5. The model processes the request; if it thinks by default, it may spend all tokens on reasoning and return an empty reply.
• 6. The app observes slower or silent responses when max_tokens is tight.
• 7. Update the regex to exclude "-flash-lite" variants using a negative lookahead.
• 8. Add unit tests to confirm reasoning is not sent to Flash‑Lite and non‑reasoning models are untouched.
• 9. Adjust gateway to treat empty replies as failures rather than successful 200 responses.

Evidence

The gateway checks the model name and, for models that reason, adds a cap.

Source

The Gemini 2.5 models I had in mind think by default, so gemini-2\.5 looked right.

Source

With a roomier budget the answer can still fit after the thinking, so the failure is quieter: the reply just arrives later than it should, from the model you picked because it was fast.

Source

The fix was one negative lookahead: return /gpt-oss|deepseek-r1|qwq|magistral|gemini-2\.5(?!-flash-lite)/i.test(model);

Source