Firecrawl vs Tavily for an AI agent: 97% vs 82% fact recall, and why retrieval breadth decides grounding

Description and use

Firecrawl delivers higher fact recall (97%) than Tavily (82%) but at higher latency and cost. Limiting source breadth causes hallucinations due to citing unretrieved pages. Choice depends on fact coverage, speed, and cost needs.

Problem

Choosing a web‑scraping backend for an AI competitor‑research agent that must reliably retrieve and cite facts from company websites.

Outcome

Firecrawl achieved higher fact recall (97% vs 82%) but at the expense of higher latency (1,391 ms vs 250 ms) and higher credit cost. Boilerplate filtering in Firecrawl can remove critical content, and limited source breadth caused the agent to cite unretrieved pages, highlighting retrieval breadth as key to grounding.

Workflow

• 1. Select 10 companies and 20 pages covering diverse site types.
• 2. Create an answer key of facts (pricing tiers, product names, taglines, YC batch) and audit against raw HTML.
• 3. Run Experiment A: fetch each page twice without an LLM, score fact recall for Tavily and Firecrawl.
• 4. Run Experiment B: execute the full agent with Gemini, constant prompts and temperature, varying the number of sources consulted.
• 5. Record fetch success, fact recall, latency, content size, boilerplate impact, and citation rejection rates.
• 6. Analyze results to compare retrieval quality, cost, and grounding.
• 7. Document findings and reproduce the experiment in the repo.

Evidence

Firecrawl found more of the facts that matter. It was about 5.6x slower, and by published pricing about 5x the credit cost per page.

Source

Every low-source run came from one configuration of mine: Firecrawl search with full-page scraping, capped at 3 results per search to save credits.

Source

The fabrication check caught every one of those citations. None reached a final report.

Source