Trust, but Verify: How I Test n8n LLM Workflows Before They Touch Production

Description and use

A deterministic test harness for n8n LLM workflows validates grounding, fallback, idempotency, and cost limits before deployment.

Problem

Deploying LLM workflows without tests risks silent failures, duplicate work, and cost overruns.

Outcome

The testing harness ensures mechanical guarantees—grounding, fallback, idempotency, and cost control—are met before production deployment, reducing silent failures and unexpected expenses.

Workflow

• Run three canned transcripts (Clean, Messy, Adversarial) to validate grounding and output correctness.
• Force‑fail each LLM provider individually and together to test fallback and loud failure paths.
• Manually crash the workflow mid‑run and rerun to verify idempotency and no duplicate extraction.
• Run the Messy fixture to check escalation rate and per‑meeting cost against budget limits.
• Ship the workflow after all checks pass.

Evidence

Everything starts with three canned transcripts, checked into the repo next to the workflow JSON: Clean, Messy, Adversarial.

Source

I keep a small table of canned outputs — valid quote, paraphrased quote (must drop), quote from a different meeting (must drop), missing quote (must drop). The gate is pure string matching, so this test is fully deterministic.

Source

Kill provider A → the workflow must fall back to provider B and the run completes.

Source

Kill the execution mid‑run — literally stop the n8n execution while it's halfway through a meeting — then rerun the same transcript.

Source

Before any new workflow version goes live, I run the messy fixture through it and check two numbers against the budgets from the cascade post: Escalation rate and Per‑meeting cost ceiling.

Source