Run a watchdog that doesn't live in the cloud it watches

Описание и применение

Самостоятельный мониторинг на локальном оборудовании обеспечивает независимый источник наблюдения, позволяя обнаруживать сбои даже при отказе облачных сервисов.

Задача

Monitoring services often run on the same cloud infrastructure they monitor, causing outages when that infrastructure fails, leaving teams unable to detect or respond to failures.

Результат

Teams gain an always‑available watchdog that can detect outages even when cloud monitoring services are down, improving incident response time and reliability.

Рабочий процесс

• Deploy a single Docker container on local hardware
• Configure the container to check internet uplink against multiple anchors
• Add DNS checks for critical domains
• Set up HTTP assertions for public services (status code, keyword in body)
• Monitor TLS certificate and domain expiry
• Include local infrastructure checks (containers, hypervisor, NAS, temperatures)
• Configure alerts to use existing notification channels

Доказательства

When AWS's us-east-1 went down in October, and Cloudflare followed in November, plenty of teams learned that their monitoring lived on the same infrastructure as the things it monitored.

Источник

The fix is old-fashioned and cheap: keep one monitor that depends on nothing but a box in your own building.

Источник