AI blog drafts fact-checked: my drafter invented a bug, 8 of 9 needed fixes

Описание и применение

Публикация AI‑генерируемых блог‑постов сопровождалась ошибками и вымышленными фактами. После проверки каждой строки против кода, логов и заметок были исправлены все недочёты, добавлены проверки на пустой контент и исправлены ошибки в протоколе Wyoming. Теперь каждый черновик проходит строгую проверку перед публикацией.

Задача

AI-generated blog drafts contained fabricated facts and errors, undermining content accuracy.

Результат

The pipeline now reliably produces accurate drafts; 9 drafts were audited, 8 required fixes, and all were corrected before publication.

Рабочий процесс

• 1. Generate draft via AI model using script and logs.
• 2. Run format and leak checks (title length, description length, tags).
• 3. Review draft claim by claim against source code, logs, and notes.
• 4. If a claim is false, edit the draft and add a correction note with date.
• 5. Queue corrected draft for publishing.
• 6. Maintain a checklist of predictable categories (e.g., "I tried X", fixes, counts, access statements).
• 7. Use canary tests to detect silent failures (e.g., tracing service address changes).

Доказательства

Six days later I fact-checked every draft sitting in the review folder against the code, the logs and my own notes.

Источник

The drafter had written a post about my podcast pipeline, which turns an article into a two-voice episode using Piper text-to-speech over the Wyoming protocol.

Источник

None of those four things happened. The real bugs were two, and both are in the code.

Источник

The speed figure was invented too. What I actually measured was 8.4 seconds of audio in 2.9 seconds of wall time, a real-time factor of about 0.35.

Источник

I rewrote the post from the code and published it the same day as Piper TTS over Wyoming: the frame format and empty model that broke my podcast.

Источник

The nine drafts have been fixed and queued one a day.

Источник