Let the sitemap decide which URLs your bot is allowed to announce

说明与用途

机器人基于账本发布URL，导致404。改用sitemap.xml并检查HTTP状态后，只发布真实存在的页面。

问题

A bot announced URLs based on a ledger of slugs, leading to 404s when builds failed or deployments were incomplete.

结果

The bot now reliably announces only published articles, eliminating 404s and ensuring announcements reflect actual live content.

工作流

• Take today’s slugs from state.json
• Fetch https://<site>/sitemap.xml and parse <loc> values
• Find <loc> containing the slug; only those become candidates
• If no candidates, report and drop the site
• Optionally fallback to same‑day <lastmod> entries with a 50% ratio guard
• Perform an HTTP request to evaluate liveness (200‑399 ok, 400‑599 definitive broken)

证据

The bot built announcement URLs from my own bookkeeping…

来源

Only those become announcement candidates.

来源

The mechanism was behaving exactly as specified.

来源