I Ran 6 n8n AI Workflows on a 1.8B Local Model: 192 Runs, 0 Broken JSON, 7 Fake Hot Leads

Description and use

Running 6 n8n workflows on a local 1.8 B CPU model produced valid JSON but 4 of 6 workflows returned incorrect content. Guard nodes caught many failures but not all.

Problem

Running n8n AI workflows on a small local 1.8B model produced many content failures while JSON output remained valid.

Outcome

0/192 responses were invalid JSON; however, 4/6 workflows produced incorrect content (e.g., lead scoring contradictions, wrong invoice dates, fabricated FAQ answers). Guard nodes mitigated many but not all issues.

Workflow

• 1. Configure n8n with local Ollama endpoint
• 2. Run each workflow 3 times per test case
• 3. Capture raw and guarded outputs via Python script
• 4. Evaluate failures per workflow
• 5. Apply guard nodes to validate content
• 6. Analyze results and propose fixes

Evidence

0 of 192 responses were invalid JSON.

Source

Score and tier disagreed in 13/18 raw runs.

Source

Wrong date 5/18, wrong total 3/18, made‑up invoice number 3/18

Source

Answered a question that isn't in the knowledge base with a confident "No, we don't" 3/3 times

Source