Building a Resilient, Claude-Powered Web Scraper with YAML (Self-Hosted)

Описание и применение

HYAE – самодостаточный скриптер, использующий YAML и Claude AI для устойчивого извлечения данных с веб‑страниц, автоматически адаптирующийся к изменениям разметки.

Задача

Traditional web scrapers break when site layouts or class names change, forcing developers to spend time fixing selectors; cloud-based AI scrapers are costly and capped.

Результат

HYAE provides a resilient, YAML-driven scraping workflow that adapts to site changes without manual selector updates, while remaining self-hosted and cost-effective.

Рабочий процесс

• Define a YAML schema with desired output fields
• Run HYAE to fetch pages respecting robots.txt and rate limits
• Claude parses raw HTML and outputs structured JSON or CSV

Доказательства

If you've maintained web scrapers at scale, you know the routine: a site updates its layout, class names change, and your XPath or CSS selectors instantly break.

Источник

HYAE is a self-hosted, YAML-driven scraping engine powered by Claude AI.

Источник

Define a simple YAML file specifying the fields you want to extract (e.g., product titles, prices, specs).

Источник

You only pay for the exact tokens you use without recurring platform markups.

Источник