I Ran 6 n8n AI Workflows on a 1.8B Local Model: 192 Runs, 0 Broken JSON, 7 Fake Hot Leads

Описание и применение

Запуск 6 рабочих процессов n8n на локальной модели 1.8 B CPU показал, что хотя JSON‑ответы были валидны, 4 из 6 процессов выдавали неверный контент. Гард‑ноды уложились в часть ошибок, но не все.

Задача

Running n8n AI workflows on a small local 1.8B model produced many content failures while JSON output remained valid.

Результат

0/192 responses were invalid JSON; however, 4/6 workflows produced incorrect content (e.g., lead scoring contradictions, wrong invoice dates, fabricated FAQ answers). Guard nodes mitigated many but not all issues.

Рабочий процесс

• 1. Configure n8n with local Ollama endpoint
• 2. Run each workflow 3 times per test case
• 3. Capture raw and guarded outputs via Python script
• 4. Evaluate failures per workflow
• 5. Apply guard nodes to validate content
• 6. Analyze results and propose fixes

Доказательства

0 of 192 responses were invalid JSON.

Источник

Score and tier disagreed in 13/18 raw runs.

Источник

Wrong date 5/18, wrong total 3/18, made‑up invoice number 3/18

Источник

Answered a question that isn't in the knowledge base with a confident "No, we don't" 3/3 times

Источник