I lost track of my own data across a homelab — so I built a whole-fleet disk analyser
Описание и применение
Автор создал Fathomline — инструмент для анализа дисков в домашнем кластере, объединяющий метаданные с нескольких хостов, точно определяющий дубликаты и обеспечивающий безопасное удаление без передачи данных в облако.
Задача
The author could not locate where data lived across multiple machines, leading to wasted space and risk of accidental deletion.
Результат
Unified view of a multi‑host estate, accurate duplicate detection, safe cleanup workflows, and no data leaves the network.
Рабочий процесс
- Deploy a read‑only agent on each host (containerised, limited capabilities).
- Agent walks filesystem, collects metadata, pushes to central PostgreSQL/SQLite via mTLS.
- Central catalogue aggregates metadata, performs progressive dedup (size → head/tail → full BLAKE3).
- UI presents estate‑wide treemaps, duplicate groups, growth trends.
- When cleanup is needed, create a dry‑run plan, sign with Ed25519, quarantine files, re‑check hashes, then delete safely.
- Optional AI concierge and sandboxed previews run locally without sending data outside.
- Repeat scans nightly or on-demand via Scan Now.
- Monitor capacity via Notification Center.
- Use setup wizard for multi‑host configuration (CA, mTLS, enrolment).
Доказательства
A lightweight, read‑only agent runs on each host, walks the filesystem, and pushes file metadata over mutually‑authenticated TLS into a central catalogue — PostgreSQL in production, SQLite for staging and single‑host setups.
Content‑verified dedup, done as a ladder. Hashing every byte of tens of millions of files is wasteful, so dedup is progressive: bucket by size → sample head/tail → confirm with a full‑content BLAKE3 hash.
- Дата записи: 2026-09-08T04:23:28.837Z