Teaching a 7B model on a CPU to finish what it starts

Description and use

The refined agent completes a variety of web‑browsing tasks on a CPU‑only machine, achieving consistent finish times (e.g., 13 turns for documentation research, 4 turns for a factual question) and reducing wasted turns through code‑enforced rules and prompt optimization.

Problem

Running a 7B language model on a CPU to perform web‑browsing tasks efficiently and reliably.

Outcome

The refined agent completes a variety of web‑browsing tasks on a CPU‑only machine, achieving consistent finish times (e.g., 13 turns for documentation research, 4 turns for a factual question) and reducing wasted turns through code‑enforced rules and prompt optimization.

Workflow

• 1. Define a single Python file that receives the objective, current findings, recent actions, and current page, and outputs a JSON action.
• 2. Implement five actions: SEARCH, NAVIGATE, READ_MORE, DISCOVER, FINISH, constrained by a JSON schema in LM Studio.
• 3. Enforce rules in code: prevent repeated searches, duplicate URLs, empty findings, and duplicate findings; return a refusal with a reason when a rule is violated.
• 4. Optimize prompt construction: keep a growing conversation, append only new content, and rebuild only when approaching the context limit; reorder prompt to share longer prefixes for faster turns.
• 5. Add progress and objective checks: after four turns without a new finding, compare findings to the objective; every eight turns, re‑check the objective; confirm before leaving a page without recording.
• 6. Separate answer generation: after FINISH, invoke a fresh model call that receives only the objective and numbered findings to produce the final answer, ensuring no hallucinations or unintended opinions.
• 7. Test across six task types (documentation research, opinion synthesis, factual question, comparison, list, how‑to) to validate robustness.
• 8. Iterate on rules based on failures: enforce fresh prompts for answer generation, flag missing numbers, limit opinions, and require evidence for each objective part.
• 9. Deploy the agent via pip install localagentbrowser and run on the target CPU.
• 10. Monitor performance metrics (turn count, time per turn) and adjust prompt ordering and context usage to maintain efficiency.

Evidence

Early versions of this agent never finished anything. They searched the same thing over and over, recorded nothing, wandered off topic and ran until the turn limit.

Source

Four turns, about twelve minutes. Not impressive for a frontier model in the cloud. But this ran on an Intel i5-9500 (six cores, no graphics card) with Qwen2.5-7B-Instruct, quantized to 4 bits and served by LM Studio.

Source

A small model reads about 8 tokens per second, and reading the prompt is about 90% of every model call.

Source

With those, version 1.27 finished a documentation research task on its own: 13 turns, about 36 minutes, with the commands and file layout it was asked for.

Source