Can a local LLM replace Claude Code? The benchmark everyone quotes is the wrong one.

Описание и применение

Модели LLM могут генерировать корректный код, но не способны надёжно выполнять роль агента, выдавая валидные вызовы инструментов, поддерживая долгосрочную последовательность и обрабатывая большие контексты. В 2026 году Mixture‑of‑Experts и локальный API позволяют запускать модели на потребительских GPU, но выбор модели и настройка контекста критичны для успешной автоматизации редактора.

Задача

Local LLMs can generate correct code but fail to reliably act as agents that emit valid tool calls, maintain long‑horizon consistency, handle large context, and follow instructions, making them unsuitable for real‑world editor automation.

Результат

A single 24GB GPU running Qwen 3.6 27B provides the sweet spot for reliable local coding agents, offering ~77% SWE‑bench score and robust tool‑calling, while smaller GPUs can use GLM‑4.7‑Flash or Gemma 4 for single‑file help.

Рабочий процесс

• Identify GPU VRAM and desired agentic capabilities
• Choose a model family and size (e.g., Qwen 3.6 27B for 24GB GPU)
• Pull the model with Ollama (e.g., ollama pull qwen3-coder:30b)
• Set environment variables to point Claude Code to localhost (export ANTHROPIC_BASE_URL="http://localhost:11434", export ANTHROPIC_AUTH_TOKEN="local-dummy")
• Run Claude Code (claude) or Codex CLI with --oss flag
• Configure context window to at least 32K or 64K
• Verify tool‑call reliability and consistency in practice
• Adjust model or harness if tool‑call failures occur

Доказательства

SWE‑bench measures "given a bug, produce a correct patch." That's a *writing* task.

Источник

Tool‑calling is a separate trained skill from writing code.

Источник

The curve climbs steeply up to ~24GB and then flattens — the 480B coder even dips below the 27B on Verified while costing 3× the memory.

Источник