Ornith-1.0-9B vs. Qwen3.5 vs. Gemma4: A Local AI Battle Royale

Description and use

A local benchmark of Ornith‑1.0‑9B against Qwen3.5‑9B and Gemma4‑12B shows that specialization improves JSON output brevity but introduces subtle bugs in code fixes and shell‑scripting.

Problem

Assess whether post‑training for agentic coding actually improves a model’s practical coding and tool‑calling performance on a laptop.

Outcome

Ornith’s fine‑tuning yields a concise JSON output and matches the base model on logic tasks, but it fails on a subtle off‑by‑one bug in a code‑fix prompt and repeats a shell‑scripting filename‑space bug present in Qwen3.5.

Workflow

• Pull GGUF models via Ollama
• Configure same quantization (Q4_K_M) for all models
• Run each model through five identical prompts with temperature 0.2
• Unload a model before loading the next to avoid CPU contention
• Collect and compare outputs for correctness, speed, and token count

Evidence

Ornith is a 9B model post‑trained specifically for agentic coding — terminal use, tool‑calling, SWE‑Bench‑style tasks.

Source

All three correctly diagnosed the off‑by‑one (range(len(nums) - k)) excludes the final window.

Source

Ornith rewrote it as a rolling‑sum optimization… it returns 20 instead of 11.

Source

Gemma4: find /var/log -type f -size +50M -exec du -h {} + | sort -hr — handles filenames with spaces correctly.

Source

All three got the values right, but Ornith's answer was the tightest: {"name":"Maria Gonzalez","age":34,"city":"Austin"} in 16 tokens.

Source