Firecrawl vs Tavily for an AI agent: 97% vs 82% fact recall, and why retrieval breadth decides grounding

Описание и применение

Firecrawl обеспечивает более высокую точность фактов (97%) по сравнению с Tavily (82%), но при этом значительно медленнее и дороже. Ограничение количества источников приводит к «галлюцинациям» из‑за цитирования нераспознанных страниц. Выбор зависит от требований к покрытию фактов, скорости и стоимости.

Задача

Choosing a web‑scraping backend for an AI competitor‑research agent that must reliably retrieve and cite facts from company websites.

Результат

Firecrawl achieved higher fact recall (97% vs 82%) but at the expense of higher latency (1,391 ms vs 250 ms) and higher credit cost. Boilerplate filtering in Firecrawl can remove critical content, and limited source breadth caused the agent to cite unretrieved pages, highlighting retrieval breadth as key to grounding.

Рабочий процесс

• 1. Select 10 companies and 20 pages covering diverse site types.
• 2. Create an answer key of facts (pricing tiers, product names, taglines, YC batch) and audit against raw HTML.
• 3. Run Experiment A: fetch each page twice without an LLM, score fact recall for Tavily and Firecrawl.
• 4. Run Experiment B: execute the full agent with Gemini, constant prompts and temperature, varying the number of sources consulted.
• 5. Record fetch success, fact recall, latency, content size, boilerplate impact, and citation rejection rates.
• 6. Analyze results to compare retrieval quality, cost, and grounding.
• 7. Document findings and reproduce the experiment in the repo.

Доказательства

Firecrawl found more of the facts that matter. It was about 5.6x slower, and by published pricing about 5x the credit cost per page.

Источник

Every low-source run came from one configuration of mine: Firecrawl search with full-page scraping, capped at 3 results per search to save credits.

Источник

The fabrication check caught every one of those citations. None reached a final report.

Источник