Benchmarking Qwen2.5 Local LLM Sizes in n8n on Apple Silicon

说明与用途

在 M2 Mac 上对五种 Qwen2.5 大小进行 n8n 基准测试，结果显示 1.5B 和 3B 模型在速度和准确性上优于推荐的 14B，尤其适用于分类、提取和工具调用工作流。

问题

Choosing the right LLM size for n8n workflows on an M2 Mac can lead to significant latency and accuracy trade‑offs.

结果

The 1.5B model outperformed the 14B on tool‑calling accuracy and speed, while the 3B model matched or exceeded 14B on extraction. 7B was slower and no better in accuracy. 0.5B was fast but too inaccurate for production routing.

工作流

• 1. Pull five Qwen2.5 models via Ollama
• 2. Set up n8n in Docker on the same Mac
• 3. Create five n8n workflows (classification, extraction, summarization, RAG Q&A, tool‑calling)
• 4. Run each workflow 5 times with each model size
• 5. Record wall‑time and accuracy metrics
• 6. Analyze results and recommend model sizes

证据

The 1.5B model beat the 14B on tool‑calling accuracy AND speed

来源

The 7B is the worst pick. It’s slow (4‑9s) and its accuracy is identical to or worse than 3B

来源