Ornith-1.0-9B vs. Qwen3.5 vs. Gemma4: A Local AI Battle Royale

Описание и применение

Оценка локального сравнения модели Ornith‑1.0‑9B с её базой Qwen3.5‑9B и Gemma4‑12B выявила, что специализация улучшает краткость JSON‑вывода, но приводит к ошибкам в исправлении кода и обработке файлов с пробелами.

Задача

Assess whether post‑training for agentic coding actually improves a model’s practical coding and tool‑calling performance on a laptop.

Результат

Ornith’s fine‑tuning yields a concise JSON output and matches the base model on logic tasks, but it fails on a subtle off‑by‑one bug in a code‑fix prompt and repeats a shell‑scripting filename‑space bug present in Qwen3.5.

Рабочий процесс

• Pull GGUF models via Ollama
• Configure same quantization (Q4_K_M) for all models
• Run each model through five identical prompts with temperature 0.2
• Unload a model before loading the next to avoid CPU contention
• Collect and compare outputs for correctness, speed, and token count

Доказательства

Ornith is a 9B model post‑trained specifically for agentic coding — terminal use, tool‑calling, SWE‑Bench‑style tasks.

Источник

All three correctly diagnosed the off‑by‑one (range(len(nums) - k)) excludes the final window.

Источник

Ornith rewrote it as a rolling‑sum optimization… it returns 20 instead of 11.

Источник

Gemma4: find /var/log -type f -size +50M -exec du -h {} + | sort -hr — handles filenames with spaces correctly.

Источник

All three got the values right, but Ornith's answer was the tightest: {"name":"Maria Gonzalez","age":34,"city":"Austin"} in 16 tokens.

Источник