A third of the measurement went dark and the report stayed green

说明与用途

由于模型已弃用，三台 AI 引擎之一返回 404，但引用报告仍为 0%。通过直接探测供应商、切换默认模型并更新摘要逻辑，系统现在能正确标记盲测，防止静默故障。

问题

A deprecated AI model caused one of three answer engines to return 404 for three weeks, but the monitoring report still showed 0% citations, masking the partial outage.

结果

The monitoring system now correctly distinguishes between zero measurements and blind measurements, preventing silent outages from going unnoticed and ensuring accurate visibility reports.

工作流

• 1. Identify that openai-search engine returned 404 due to deprecated gpt-4o-search-preview.
• 2. Probe the provider directly instead of relying on configuration or catalog listings.
• 3. Change the default model in the tracker source to a non-deprecated model.
• 4. Update summarize() to name any engine that held a valid key but measured zero probes as BLIND.
• 5. Deploy changes, restart the process, and verify live run metrics.
• 6. Commit changes to main to ensure reproducibility.

证据

OpenAI deprecated gpt-4o-search-preview, so every request on the openai-search leg returned HTTP 404.

来源

Changed the default model in the tracker source rather than patching an env var on the box, so a rebuild cannot lose it.

来源