How to Fit More Web Search Data into a Local LLM

说明与用途

使用 SerpApi 的 Markdown 输出与 JSON 限制器相结合，使本地 LLM 能够处理更多搜索数据，节省 27% 令牌。

问题

Local LLMs can only use web search results that fit in their context window, but a single SerpApi response often exceeds that window.

结果

Using Markdown output with a looser JSON Restrictor allows a local LLM to ingest more search data within its context window, reducing token usage by 27% on average across seven SerpApi tools.

工作流

• 1. Configure the SerpApi request with the desired output format (JSON or Markdown) and the json_restrictor parameter.
• 2. If using Markdown, add output=md to the request URL.
• 3. The SerpApi server applies the json_restrictor to trim the response, then renders the remaining data as Markdown.
• 4. The local LLM receives the trimmed Markdown (or JSON) and processes it within its 32K context window.
• 5. Benchmark the payload size and end‑to‑end performance for each output format to determine the optimal configuration.

证据

SerpApi's new Markdown output changes that budget. The same data takes fewer tokens in Markdown than in JSON, so the restrictor can keep more of it.

来源

The app now lets you choose what the model reads after each search. The benchmark compares four outputs, and a model only ever reads three of them.

来源

To see what the format alone saves, we kept the restrictor strings from the first blog and sent the same seven searches as JSON and as Markdown. The seven results took 4,668 tokens in JSON and 3,412 in Markdown, 27% fewer.

来源