Monitoring YouTube without the Data API (and without the quota headache)

说明与用途

脚本解析公开的 YouTube 页面，提取所有视频组件，无需 API 密钥或配额限制，使用 7 天内存去重，仅输出新视频。

问题

Relying on the YouTube Data API for frequent monitoring causes quota limits and slow approval for increases, making it impractical for real‑time alerts on new videos.

结果

The scraper reliably retrieves all public videos for a keyword, including Shorts, without API keys or quota limits, and deduplicates results to provide true "new" alerts.

工作流

• 1. Send HTTP GET to youtube.com/results?search_query={keyword} without authentication.
• 2. Parse the embedded ytInitialData JSON blob from the page source.
• 3. Iterate over known renderer components (videoRenderer, gridVideoRenderer, compactVideoRenderer, lockupViewModel, shortsLockupViewModel).
• 4. Extract video ID, title, channel, view count, approximate publish time, thumbnail URL, and Shorts duration if available.
• 5. Detect blocked responses by checking if response size < 50,000 bytes and rotate IP if necessary.
• 6. Store video IDs in a 7‑day memory keyed by keyword set.
• 7. On each scheduled run (e.g., every 15 minutes), emit only videos not seen in the memory.
• 8. Send new video data to the chosen alerting channel (e.g., webhook).

证据

Polling five keywords every fifteen minutes would need 480 searches a day. You would be rate‑limited before lunch.

来源

YouTube does not use one component. It uses several, and which one you get depends on the page and on whatever A/B test you have been bucketed into:

来源

When YouTube does not like your IP it returns HTTP 200 with a much smaller page, not a 4xx.

来源