SerpApi Achieves 97.6% Accuracy on SimpleQA Verified

说明与用途

SerpApi 与 GPT‑6.1 Sol 结合，在 SimpleQA Verified 基准上实现了 97.6% 的准确率，使用 Google Search API 的 Markdown 输出和 Extract API 进行页面内容提取。

问题

How to evaluate AI agents’ ability to answer factual questions using web search and extraction APIs.

结果

SerpApi’s combined search and extraction approach achieved 97.6% accuracy on SimpleQA Verified, matching Tavily’s score and topping other providers’ reported accuracies.

工作流

• 1. Connect SerpApi Google Search API (engine=google) with Markdown output (output=md) to GPT‑6.1 Sol.
• 2. Use SerpApi Extract API (engine=extract, mode=scrape, include_markdown=true, json_restrictor=text_content.markdown) to fetch page content when needed.
• 3. Allow multiple searches and fetches per question; limit each page fetch to 48,000 characters and each model input to 56,000 tokens.
• 4. Let the model decide when to search or read a page within these limits.
• 5. Grade final answers with GPT‑5.6 Sol (high reasoning) without tool access.
• 6. Exclude any responses where search results exposed benchmark answers or hints (40 questions).

证据

We excluded 40 questions where we detected exposure to public benchmark answers or guidance.

来源

The model answered 937 of the remaining 960 questions correctly, giving us 97.6% accuracy.

来源

We used the regular SerpApi Google Search API with Markdown output.

来源

We used engine=extract with mode=scrape and returned the page's Markdown through include_markdown=true and json_restrictor=text_content.markdown.

来源