AgentCrawl, a small self-hosted crawler for AI agents

**Brief Summary (2‑3 sentences)** AgentCrawl is a lightweight, self‑hosted web crawler designed specifically for AI agents. It lets developers quickly add web‑r