My AI-Built Site's Internal Links Were Unmanageable — So I Mapped the Whole Graph Before Touching Anything

说明与用途

AI 构建的网站因页面孤立生成导致内部链接管理失控。自定义爬虫生成完整链接图，识别孤立页、重复页和中心页，随后实施系统化链接合并与重定向，提升站点结构。

问题

Internal linking was unmanageable because AI-generated pages were created in isolation without a global link ledger, leading to competing pages, orphans, dead ends, and unclear link strategy.

结果

The tool revealed 4,868 pages with 3,931 product URLs, reconstructed 268 category groups covering 92% of products, and enabled a systematic link consolidation workflow that reduced orphan pages and concentrated link equity on stronger pages.

工作流

• 1. Crawl site and separate navigation from content links using a 90% threshold rule.
• 2. Export graph as CSV, GEXF, GraphML, JSON, and a single offline HTML report.
• 3. Join pages with Search Console data to assess traffic and competition.
• 4. Identify merge candidates and redirect weaker pages.
• 5. Re-point links toward ranking pages and fix orphans/dead ends.
• 6. Re-crawl and diff edge lists to verify consolidation.

证据

internal linking is the one lever AI never managed, because it's site-wide state, not page state.

来源

I built the instrument I needed, and open-sourced it: internal-links-export.

来源

The tool recovered the de-facto taxonomy from the link structure itself: every non-hub page grouped under its strongest content in-link source.

来源

With the graph exported (pages CSV + edge list CSV), the workflow I'm running is: Join pages against Search Console, Find merge candidates, Re-point links, Fix the floor, Re-crawl and diff.

来源