A client pinged me on a Saturday morning: "your site is down."

Описание и применение

Клиент сообщил о недоступности сайта, но проблема была в DNS‑неработоспособности, которую обычные мониторы не видели. Внедрённые проверки DNS, резервный провайдер и план восстановления за 10 минут спасли ситуацию, но доверие клиентов пострадало.

Задача

A client reported site downtime, but the issue was DNS failure, not the application.

Результат

The client restored DNS service within 10 minutes, preventing further customer impact, but trust was damaged due to delayed communication.

Рабочий процесс

• Confirm scope: dig failing names against public resolvers and ISP resolver.
• Decide path: if provider down, switch NS to second provider; if own mistake, restore exported zone to primary.
• Execute runbook: use exact NS strings, export filename, and verification command.
• Verify and communicate: test from mobile data, post status update.

Доказательства

The thing that broke sits one layer above every request they serve, so every tool they own said everything was fine while every customer saw a "server not found" page.

Источник

The 10‑minute restore path, as a checklist: ...

Источник