Can Local LLMs Actually Call Functions? I Tested 2 Models on 6 Real API Tasks

Description and use

Benchmarking local LLMs for function calling revealed Llama‑3.2‑3B excels in speed and JSON validity, while Qwen3‑14B outperforms in argument accuracy, multi‑turn handling, and error recovery, guiding model choice for coding agents.

Problem

Local LLMs need to reliably produce valid JSON tool calls with correct arguments for real-world APIs, but existing claims are unverified.

Outcome

Llama‑3.2‑3B: 98% JSON validity, 24 tok/s, 0.4 s first‑token, 4.1 GB RAM – best for high‑speed, low‑cost agents. Qwen3‑14B: 95% JSON validity, 5 tok/s, 1.2 s first‑token, 9.3 GB RAM – best for multi‑turn accuracy and error recovery.

Workflow

• 1. Clone benchmark repo
• 2. Install dependencies and pull models via Ollama
• 3. Run benchmark with --models all
• 4. Review JSON results and agent traces

Evidence

Both models cleared 95% on basic JSON validity. Llama‑3.2‑3B actually edged out Qwen3‑14B here — the smaller 3B model has a tighter tool‑calling format that's more reliable in practice.

Source

Qwen3‑14B extracts the right values from natural language 82% of the time vs 71% for the 3B model. On multi‑argument tasks Qwen3 wins by 11 points. On multi‑turn context Qwen3 wins by 20 points.

Source

Qwen3 recovers correctly 65% of the time. Llama‑3.2‑3B recovers 48% of the time and hallucinates a "fix" 31% of the time.

Source