My self-healing homelab saved 17 seconds. Here's why I kept it

Description and use

<p>A power cut rebooted my home servers one June morning, and the reverse proxy didn't come back. Every site I run goes through it, and I only found out that evening. The alert had fired on time, but it reached my phone through a public URL, and that URL went through the proxy that was down.</p> <blockquote> <p><strong>TL;DR</strong> I built an auto-remediation chain for my homelab: Prometheus → Alertmanager → n8n → a policy gate → an executor that can only run actions from a closed list. Restarting a service is one line. The real work was teaching the bot when to refuse. Then I timed it again

Problem

<p>A power cut rebooted my home servers one June morning, and the reverse proxy didn't come back. Every site I run goes through it, and I only found out that evening. The alert had fired on time, but it reached my phone through a public URL, and that URL went through the proxy that was down.</p> <blockquote> <p><strong>TL;DR</strong> I built an auto-remediation chain for my homelab: Prometheus → Alertmanager → n8n → a policy gate → an executor that can only run actions from a closed list. Restarting a service is one line. The real work was teaching the bot when to refuse. Then I timed it again

Outcome

<p>A power cut rebooted my home servers one June morning, and the reverse proxy didn't come back. Every site I run goes through it, and I only found out that evening. The alert had fired on time, but it reached my phone through a public URL, and that URL went through the proxy that was down.</p> <blockquote> <p><strong>TL;DR</strong> I built an auto-remediation chain for my homelab: Prometheus → Alertmanager → n8n → a policy gate → an executor that can only run actions from a closed list. Restarting a service is one line. The real work was teaching the bot when to refuse. Then I timed it again

Evidence

My self-healing homelab saved 17 seconds. Here's why I kept it

Source