Deploy, backup and monitor a Django app on two servers

Описание и применение

Автор описал, как после обновления ОС на сервере приложения сайт был недоступен пять часов из‑за ошибок запуска процессов. Внедрив структуру развертывания с отдельными папками релизов, атомарный переключатель symlink и автоматический откат при ошибке, а также перенес резервное копирование на отдельный мониторинговый сервер с read‑only ключом, он устранил подобные сбои и улучшил наблюдаемость.

Задача

The author upgraded the Ubuntu OS on the app server from 24.04 to 26.04, causing the site to be unavailable for five hours due to process failures and memory exhaustion.

Результат

The new deployment and backup strategy prevented a five‑hour outage, reduced memory pressure, and provided automated health reporting that correlates metrics with code changes.

Рабочий процесс

• Dump the database
• Create a release worktree and link shared files
• Build a virtualenv if requirements changed
• Run migrations and collectstatic from the new release while the old one serves
• Atomically switch current to the new release
• Restart gunicorns and Daphnes one at a time
• Run a smoke test and revert if any request returns 500

Доказательства

The upgrade took 20 minutes. The site came back five hours later.

Источник

Every deploy is its own folder: /var/www/html/releases/<sha>/

Источник

Backups are pulled nightly at 02:00 UTC and before every deploy. At 03:00 the monitoring box pulls the dumps.

Источник

The monitoring box has two SSH keys on the app box, one per directory, and each is locked down in authorized_keys like this: command="rrsync -ro /var/backups/extratime/daily",restrict,from="<monitoring box's WireGuard address>" ssh-ed25519 AAAA...

Источник

Exit code 126 means the shell found the program but couldn't execute it.

Источник