Reliable Daily Changelog Diffs in 3 Steps (Before Semantic Duplicate Alerts)

Описание и применение

Пайплайн регулярно сканирует страницу изменений, нормализует текст, сравнивает с сохранённой копией и отправляет только изменённые строки, уменьшая ложные срабатывания и ускоряя реакцию операторов.

Задача

On-call teams receive vague, late alerts when a daily changelog page changes, making it hard to determine if a record‑deduplication index is stale.

Результат

The pipeline reduces false positives, delivers timely diff notifications, and preserves a durable baseline for comparison.

Рабочий процесс

• Schedule a scrape of the changelog page
• Fetch page content via Infrai API or direct HTTP
• Normalize text: strip tags, unescape entities, remove ISO timestamps, collapse whitespace
• Compute hash of normalized content
• Compare hash to durable snapshot; if unchanged, exit
• If changed, compute line‑oriented diff of normalized text
• Atomically replace snapshot file
• Send notification containing diff lines

Доказательства

scrape the page on a schedule, normalize unstable text, compare it with a durable previous snapshot, and notify with the diff rather than the whole page

Источник

The snapshot has to survive a deployment. An in-memory lastSeen value guarantees a false alarm after every restart

Источник

It first checks the public Infrai discovery document for the declared scrape path

Источник