claimcheck: Gemma 4 fact-checks my AI blog drafts against their own sources

说明与用途

claimcheck 使用 Gemma 4 对 AI 生成的草稿进行事实检查，比较其自身引用的来源，并在发现问题时阻止发布。

问题

AI‑generated drafts can contain factual errors that mislead readers, and existing fact‑checking tools are often black boxes or require paid APIs.

结果

claimcheck detected a false claim about Python 3.15, flagged a false alarm about model token limits, and showed that many claims in a popular article were unsupported due to poor linking. The tool prevented publishing of the erroneous draft and highlighted areas needing better source links.

工作流

• Extract claims from the draft using Gemma 4, ensuring each claim quotes the draft.
• Retrieve every linked source page and split it into passages.
• For each claim, use a lexical index to select the closest passages and ask Gemma 4 to judge the claim as supported, contradicted, or not_found, returning an evidence sentence.
• Verify the evidence sentence exists verbatim in the source and that all numbers in the claim appear in the evidence or surrounding text.
• Generate a report listing problems first, show draft text next to source text, and exit with code 1 if any claim is not supported.
• Use the exit code to block CI or publishing steps.
• Optional: run claimcheck on old posts to re‑check as sources change.

证据

The stdlib now returns frozendict from EnumType.__members__ and errno.errorcode instead of MappingProxyType or dict.

来源

Gemma 4 lists the checkable claims in the draft (numbers, versions, limits, defaults, "X supports Y") and has to quote the draft for each one.

来源

A full article took 3 to 5 minutes and about 70k input tokens. At DigitalOcean's listed Gemma 4 serverless rate ($0.18 per 1M input tokens and $0.50 per 1M output tokens as of October 2026), that is about 1.4 cents per article.

来源