A publish gate for AI-assisted posts: check every claim before it goes out

说明与用途

使用事实文件和LLM的发布门禁，防止AI生成内容中的未验证声明传播。

问题

AI‑generated posts can contain unverified numbers, unmeasured claims, or stock phrases that mislead readers.

结果

The gate caught unverified numbers, unmeasured claims, and stock phrases, preventing potentially misleading posts from being published. It also identified a publisher bug when a Show HN post was incorrectly marked as published.

工作流

• Create facts.md with evidence for every claim and a not‑claimable section
• Define deterministic rules for numbers, phrases, structure, and links
• Run deterministic checks before model involvement
• Send draft and facts.md to an LLM for claim verdicts (SUPPORTED/UNSUPPORTED/CONTRADICTED)
• If all claims are SUPPORTED, write a sidecar file with SHA‑256 of the post body
• Publisher script calls require_verified() to ensure sidecar matches and LLM step ran
• Publish only if verification passes; otherwise refuse and log the failure

证据

Every claim and every number a post may use lives in facts.md, each with its evidence: a file path, a command and its output, or a URL.

来源

The last check sends the draft and the facts file to a model and asks for a verdict per claim: SUPPORTED, UNSUPPORTED or CONTRADICTED.

来源

A PASS writes a small sidecar file with the SHA-256 of the post body.

来源