AI Makes Up Statistics: We Checked 186 of Its Claims

说明与用途

AI 经常通过给可信来源附加看似合理的数字来捏造统计数据，导致发布内容中出现未验证的主张。在对 20 篇 DeepSeek V4 Flash 生成的文章进行测试时，56.5% 的可检查声明没有来源，4.3% 与来源相矛盾，只有 39.2% 被确认。结果强调了手动验证和删除不可验证数字的必要性。

问题

AI-generated content frequently invents statistics, attaching plausible numbers to real or trusted sources, leading to unverified claims in published articles.

结果

The test showed that 105 of 186 checkable claims (56.5%) could not be traced to any source, 8 were contradicted, and every article contained at least one unverified claim. Only two of seven manually traced claims held up, both with minor issues.

工作流

• Generate 20 blog posts with AI using the specified prompt.
• Extract each checkable claim from the drafts.
• Search for a source that supports each claim using automated lookup services.
• Mark claims as confirmed, no source found, or contradicted.
• Flag unconfirmed claims for editor review.
• Manually trace a sample of flagged claims to verify authenticity.
• Adjust or remove unverified statistics before publishing.

证据

Six in ten. That is how many of the checkable claims in 20 AI-written blog posts I could not confirm, even with a fact-checking pipeline built for exactly that job.

来源

105 of 186 checkable claims (56.5%) could not be traced to any source, 8 more were contradicted by one, and every one of the 20 articles contained at least one of them.

来源

The model rarely invents opinions; it invents figures.

来源

Highlight every number, date and named source in the draft. Search the exact figure alongside the source's name — if the only results are other blogs repeating it, you have a zombie stat.

来源