Automating Discourse SEO with IndexNow

Description and use

The discourse-indexnow plugin automatically submits new and updated Discourse forum topics to search engines via IndexNow, speeding content discovery while respecting privacy and rate limits.

Problem

Discourse forums generate dynamic content, and search engines discover new or updated topics only after crawling, causing delays in indexing.

Outcome

The plugin enables real‑time notification of topic changes to search engines, improving discovery speed while maintaining privacy controls and respecting API limits.

Workflow

• Detect public topic creation, update, or deletion in Discourse
• Determine eligible URLs (exclude private, restricted, or unlisted content)
• Group URLs (including multilingual variants) into batches
• Apply rate limits, cooldowns, and retry logic for API requests
• Send IndexNow batch request to search engines
• Log success or failure and update admin dashboard
• Provide manual submission and backfill options via admin interface

Evidence

Discourse already has sitemaps, and they work fine. But I wanted something a little more proactive.

Source

IndexNow is an open protocol that lets websites notify participating search engines when URLs are added, updated, or deleted.

Source

One important thing to understand: IndexNow doesn't guarantee indexing or better rankings.

Source

The plugin checks whether a URL is eligible before submitting it.

Source

The plugin also splits large batches to respect IndexNow's 10,000-URL-per-request limit.

Source