Let the sitemap decide which URLs your bot is allowed to announce

Description and use

The bot was announcing URLs based on a ledger, causing 404s. Switching to sitemap.xml and HTTP status checks ensures only live pages are announced.

A bot announced URLs based on a ledger of slugs, leading to 404s when builds failed or deployments were incomplete.

The bot now reliably announces only published articles, eliminating 404s and ensuring announcements reflect actual live content.

Take today’s slugs from state.json
Fetch https://<site>/sitemap.xml and parse <loc> values
Find <loc> containing the slug; only those become candidates
If no candidates, report and drop the site
Optionally fallback to same‑day <lastmod> entries with a 50% ratio guard
Perform an HTTP request to evaluate liveness (200‑399 ok, 400‑599 definitive broken)

The bot built announcement URLs from my own bookkeeping…

Only those become announcement candidates.

The mechanism was behaving exactly as specified.