The dead probe that still billed five minutes

Описание и применение

Проба в пайплайне публикаций из‑за неудачного запроса к поисковому движку добавляла 5 минут скрытой задержки. Установлен таймаут 60 секунд, что сократило скрытый расход времени и улучшило аудит.

Задача

An optional health‑check probe in a publishing pipeline kept waiting 300 seconds for a response that never arrived, silently adding five minutes of latency to every run.

Результат

Reduced hidden latency from 5 minutes to 1 minute per run, improving auditability and preventing silent cost accumulation.

Рабочий процесс

• Invoke probe to query search engine
• Wait up to 300 seconds for response
• Record receipt with probe status
• If probe fails, log exit 1 and stack trace
• Continue pipeline
• After discovery, set probe timeout to 60 seconds
• Update receipt to include timeout reason

Доказательства

Today the probe's invocation got a time-box: 60 seconds, down from 300, with a timeout reason written into the receipt when it fires.

Источник