Benchmarking Qwen2.5 Local LLM Sizes in n8n on Apple Silicon

Описание и применение

Тестирование пяти размеров модели Qwen2.5 в n8n на Mac M2 показало, что модели 1.5B и 3B обеспечивают лучшую скорость и точность, чем рекомендованный 14B, особенно в задачах классификации, извлечения и вызова инструментов.

Задача

Choosing the right LLM size for n8n workflows on an M2 Mac can lead to significant latency and accuracy trade‑offs.

Результат

The 1.5B model outperformed the 14B on tool‑calling accuracy and speed, while the 3B model matched or exceeded 14B on extraction. 7B was slower and no better in accuracy. 0.5B was fast but too inaccurate for production routing.

Рабочий процесс

• 1. Pull five Qwen2.5 models via Ollama
• 2. Set up n8n in Docker on the same Mac
• 3. Create five n8n workflows (classification, extraction, summarization, RAG Q&A, tool‑calling)
• 4. Run each workflow 5 times with each model size
• 5. Record wall‑time and accuracy metrics
• 6. Analyze results and recommend model sizes

Доказательства

The 1.5B model beat the 14B on tool‑calling accuracy AND speed

Источник

The 7B is the worst pick. It’s slow (4‑9s) and its accuracy is identical to or worse than 3B

Источник