How to Fit More Web Search Data into a Local LLM

Description and use

Using SerpApi’s Markdown output with a JSON restrictor lets a local LLM ingest more search data, saving 27 % tokens.

Problem

Local LLMs can only use web search results that fit in their context window, but a single SerpApi response often exceeds that window.

Outcome

Using Markdown output with a looser JSON Restrictor allows a local LLM to ingest more search data within its context window, reducing token usage by 27% on average across seven SerpApi tools.

Workflow

• 1. Configure the SerpApi request with the desired output format (JSON or Markdown) and the json_restrictor parameter.
• 2. If using Markdown, add output=md to the request URL.
• 3. The SerpApi server applies the json_restrictor to trim the response, then renders the remaining data as Markdown.
• 4. The local LLM receives the trimmed Markdown (or JSON) and processes it within its 32K context window.
• 5. Benchmark the payload size and end‑to‑end performance for each output format to determine the optimal configuration.

Evidence

SerpApi's new Markdown output changes that budget. The same data takes fewer tokens in Markdown than in JSON, so the restrictor can keep more of it.

Source

The app now lets you choose what the model reads after each search. The benchmark compares four outputs, and a model only ever reads three of them.

Source

To see what the format alone saves, we kept the restrictor strings from the first blog and sent the same seven searches as JSON and as Markdown. The seven results took 4,668 tokens in JSON and 3,412 in Markdown, 27% fewer.

Source