Can a local LLM replace Claude Code? The benchmark everyone quotes is the wrong one.

Description and use

Local LLMs can write correct code but fail to reliably act as agents that emit valid tool calls, maintain long‑horizon consistency, handle large context, and follow instructions. In late 2026, Mixture‑of‑Experts models and native local API support enable running large models on consumer GPUs, but selecting the right family and configuring context is essential for real‑world editor automation.

Problem

Local LLMs can generate correct code but fail to reliably act as agents that emit valid tool calls, maintain long‑horizon consistency, handle large context, and follow instructions, making them unsuitable for real‑world editor automation.

Outcome

A single 24GB GPU running Qwen 3.6 27B provides the sweet spot for reliable local coding agents, offering ~77% SWE‑bench score and robust tool‑calling, while smaller GPUs can use GLM‑4.7‑Flash or Gemma 4 for single‑file help.

Workflow

• Identify GPU VRAM and desired agentic capabilities
• Choose a model family and size (e.g., Qwen 3.6 27B for 24GB GPU)
• Pull the model with Ollama (e.g., ollama pull qwen3-coder:30b)
• Set environment variables to point Claude Code to localhost (export ANTHROPIC_BASE_URL="http://localhost:11434", export ANTHROPIC_AUTH_TOKEN="local-dummy")
• Run Claude Code (claude) or Codex CLI with --oss flag
• Configure context window to at least 32K or 64K
• Verify tool‑call reliability and consistency in practice
• Adjust model or harness if tool‑call failures occur

Evidence

SWE‑bench measures "given a bug, produce a correct patch." That's a *writing* task.

Source

Tool‑calling is a separate trained skill from writing code.

Source

The curve climbs steeply up to ~24GB and then flattens — the 480B coder even dips below the 27B on Verified while costing 3× the memory.

Source