I Gave My AI Agents Their Own Documentation Crawler, and Pulled 60 Pages of Clean Markdown in 49 Seconds

说明与用途

pulpie-mcp MCP 服务器使 AI 代理能够在 49 秒内自动抓取、转换并保存完整的网页文档为干净的 Markdown，消除了手动步骤并保留完整内容。

问题

AI agents often lack a reliable way to fetch and store full web documentation as clean Markdown, leading to manual steps and summarized or incomplete content.

结果

In under 49 seconds, Claude crawled 60 documentation pages, saving them as clean Markdown files, with a ~10s cold start and ~37s crawl time, enabling agents to reference full docs without context window overload.

工作流

• 1. Install pulpie-mcp via uv or pip.
• 2. Add the MCP server to Claude or other clients.
• 3. Use tools like fetch_markdown, save_markdown, crawl_docs, or list_library to retrieve or manage docs.
• 4. Agents can request a full docs crawl, which saves files locally and returns metadata.
• 5. Agents can then read or reference specific Markdown files as needed.

证据

I had Claude test crawl_docs against a documentation site with 60 pages.

来源

The backend cold start was ~10s and crawling 60 pages took ~37s, total ~49s.

来源

pulpie-mcp uses feyninc/pulpie-orange-small, a 210M parameter encoder model, using ~420 MB VRAM.

来源

The MCP server starts the backend automatically and shares it across agents, shutting down after 30 minutes of inactivity.

来源