Find broken links and dead sitemap URLs without installing a crawler

Description and use

Automated link and sitemap checker using Apify Actors, scheduled weekly, outputs JSON for integration with Slack, GitHub, or spreadsheets.

Problem

Running a weekly link check on multiple sites and exporting results to a spreadsheet is cumbersome with desktop crawlers.

Outcome

The two Apify Actors perform comprehensive link checks in under a minute for small sites, returning detailed JSON that can be consumed by downstream tools.

Workflow

• 1. Deploy Broken Link Checker Actor with startUrls, maxPagesToCrawl, maxLinksToCheck.
• 2. Actor crawls pages, collects links/images, performs HEAD then GET for errors, respects robots.txt, reports 401/403/429 as warnings, lists redirects if reportRedirects enabled.
• 3. Actor outputs JSON with url, status, severity, problem, foundOn details.
• 4. Deploy Sitemap Extractor Actor with startUrls, checkStatus, maxUrlsPerSite.
• 5. Actor reads sitemaps, checks each URL, reports redirects, robots‑blocked, dead domains.
• 6. Schedule Actors weekly via Apify UI or API, using Changed since = 7 days to limit to new/changed pages.
• 7. Call Actors from code using curl or SDKs; pipe JSON output to Slack, GitHub issue, or spreadsheet.
• 8. Use provided code snippets (curl, Python, Node.js, n8n) for integration.

Evidence

Broken Link Checker crawls a site, collects every link and image on the pages it visits, and checks each URL once:

Source

On that test site, 5 pages gave 143 unique URLs and 8 broken ones in under a minute.

Source

Schedule it weekly with Changed since = 7 days to check only new or changed pages.

Source