SerpApi Achieves 97.6% Accuracy on SimpleQA Verified

Description and use

SerpApi combined with GPT‑6.1 Sol achieved 97.6 % accuracy on the SimpleQA Verified benchmark, using Google Search API with Markdown output and Extract API for page content extraction.

Problem

How to evaluate AI agents’ ability to answer factual questions using web search and extraction APIs.

Outcome

SerpApi’s combined search and extraction approach achieved 97.6% accuracy on SimpleQA Verified, matching Tavily’s score and topping other providers’ reported accuracies.

Workflow

• 1. Connect SerpApi Google Search API (engine=google) with Markdown output (output=md) to GPT‑6.1 Sol.
• 2. Use SerpApi Extract API (engine=extract, mode=scrape, include_markdown=true, json_restrictor=text_content.markdown) to fetch page content when needed.
• 3. Allow multiple searches and fetches per question; limit each page fetch to 48,000 characters and each model input to 56,000 tokens.
• 4. Let the model decide when to search or read a page within these limits.
• 5. Grade final answers with GPT‑5.6 Sol (high reasoning) without tool access.
• 6. Exclude any responses where search results exposed benchmark answers or hints (40 questions).

Evidence

We excluded 40 questions where we detected exposure to public benchmark answers or guidance.

Source

The model answered 937 of the remaining 960 questions correctly, giving us 97.6% accuracy.

Source

We used the regular SerpApi Google Search API with Markdown output.

Source

We used engine=extract with mode=scrape and returned the page's Markdown through include_markdown=true and json_restrictor=text_content.markdown.

Source