Can a local LLM replace Claude Code? The benchmark everyone quotes is the wrong one.

Description and use

Local LLMs can replace Claude Code for coding tasks when the correct model family is chosen, hardware is matched, and tool‑calling reliability is verified, but they require careful configuration and monitoring to avoid agentic failures.

Problem

Local LLMs can write correct functions but fail to reliably act as agents that emit valid tool calls, maintain long‑horizon consistency, handle large context, and follow instructions, making them unsuitable for real‑world editor integration.

Outcome

Local LLMs can replace Claude Code for coding tasks when the correct model family is chosen, hardware is matched, and tool‑calling reliability is verified, but they require careful configuration and monitoring to avoid agentic failures.

Workflow

• Identify hardware constraints and desired agentic capabilities.
• Select an appropriate model family (e.g., Qwen 3.6 27B for 24GB GPU, GLM‑4.7‑Flash for 8GB laptop).
• Pull the model via Ollama (e.g., ollama pull qwen3-coder:30b).
• Set environment variables: export ANTHROPIC_BASE_URL="http://localhost:11434" and export ANTHROPIC_AUTH_TOKEN="local-dummy".
• Run Claude Code (claude) or Codex CLI with --oss pointing to the local server.
• Ensure context window is set to at least 32K (preferably 64K) to avoid early starvation.
• Monitor tool‑call reliability and retry behavior; adjust model size if frequent malformed calls occur.
• Unset ANTHROPIC_BASE_URL to revert to cloud if needed.

Evidence

SWE‑bench measures "given a bug, produce a correct patch." That's a *writing* task.

Source

Tool‑calling is a separate trained skill from writing code.

Source

Reliability falls off a cliff below ~4B parameters.

Source

Set the context window explicitly — at least 32K, ideally 64K+.

Source