Firecrawl vs Tavily for an AI agent: 97% vs 82% fact recall, and why retrieval breadth decides grounding

说明与用途

Firecrawl 的事实召回率为 97%，高于 Tavily 的 82%，但延迟更高、成本更高。限制来源数量会导致引用未检索页面，产生“幻觉”。选择取决于对事实覆盖率、速度和成本的需求。

问题

Choosing a web‑scraping backend for an AI competitor‑research agent that must reliably retrieve and cite facts from company websites.

结果

Firecrawl achieved higher fact recall (97% vs 82%) but at the expense of higher latency (1,391 ms vs 250 ms) and higher credit cost. Boilerplate filtering in Firecrawl can remove critical content, and limited source breadth caused the agent to cite unretrieved pages, highlighting retrieval breadth as key to grounding.

工作流

• 1. Select 10 companies and 20 pages covering diverse site types.
• 2. Create an answer key of facts (pricing tiers, product names, taglines, YC batch) and audit against raw HTML.
• 3. Run Experiment A: fetch each page twice without an LLM, score fact recall for Tavily and Firecrawl.
• 4. Run Experiment B: execute the full agent with Gemini, constant prompts and temperature, varying the number of sources consulted.
• 5. Record fetch success, fact recall, latency, content size, boilerplate impact, and citation rejection rates.
• 6. Analyze results to compare retrieval quality, cost, and grounding.
• 7. Document findings and reproduce the experiment in the repo.

证据

Firecrawl found more of the facts that matter. It was about 5.6x slower, and by published pricing about 5x the credit cost per page.

来源

Every low-source run came from one configuration of mine: Firecrawl search with full-page scraping, capped at 3 results per search to save credits.

来源

The fabrication check caught every one of those citations. None reached a final report.

来源