How I stopped an LLM from inventing facts in my daily YouTube Shorts

Description and use

An automated workflow for daily YouTube Shorts where an LLM drafts scripts from verified facts, and code‑based guards ensure no invented numbers or rumors are included.

Problem

An LLM used to generate daily YouTube Shorts scripts introduced unverified numbers and rumors, compromising factual accuracy on a facts‑centric channel.

Outcome

The workflow ensures daily Shorts are factually accurate by combining verified source facts with multiple automated guards, reducing misinformation and enabling continuous, unattended publishing.

Workflow

• 1. Author writes verified facts with sources.
• 2. LLM turns facts into a script draft.
• 3. Code guard 1: number whitelist – every number must appear in the facts or be an allowed countdown; otherwise reject and retry.
• 4. Code guard 2: repetition check – reject repeated n‑grams or ideas across recent videos.
• 5. Code guard 3: rumor labeling – facts prefixed with "RUMOR:" must be presented as rumors; otherwise reject.
• 6. Code guard 4: second pass at temperature 0 to list unsupported claims; if any, reject.
• 7. Code guard 5: facts‑only fallback – if model fails repeatedly, build video directly from facts.
• 8. Voice and image pipelines have independent fallbacks to ensure continuous operation.
• 9. Publish the final Short.
• 10. Track improvements via issue #11 to add fact‑index citation checks.

Evidence

I run a channel that publishes one Short every day, unattended. The first version let the LLM write the script from a topic. It worked until it didn't: it slipped in numbers that weren't in my notes and presented rumours as facts.

Source

I write the verified facts once (with sources). The AI only turns them into a script, and the script has to pass guards written in plain code.

Source

Number whitelist. Every number in the script must appear in the facts (plus a few allowed ones, like a countdown). Anything else is rejected and the model retries.

Source

A second pass at temperature 0 that lists claims not supported by the facts.

Source

Same philosophy for the rest of the pipeline: voice (Edge TTS, then ElevenLabs, Cloudflare, offline system voice) and images (Cloudflare FLUX, NVIDIA, your own generator, archive, procedural gradient) each have a chain, so one service being down never stops the channel.

Source