Ban Naked Numbers: A Token Gate for AI-Drafted Docs

说明与用途

文章介绍了一种自动化方案，通过令牌门控和事实注册表阻止 AI 生成文档中的裸数字值。

问题

AI‑generated documentation often inserts incorrect numeric constants, causing costly errors for users.

结果

All numeric claims in AI‑drafted docs are now traceable to a verified source, eliminating accidental mis‑documentation.

工作流

• 1. Create a facts.json registry with value, unit, source, verified_at, and owner for each numeric claim.
• 2. Prompt the model to emit only tokens like {{fact:queue.backoff_ms}} instead of raw numbers.
• 3. Run claimcheck.py in two stages: draft and rendered, to ensure every token resolves to a registered fact and no naked numbers remain.
• 4. Fail the PR if any unregistered numeric claim is detected.

证据

A docs PR landed in March with one changed sentence: the queue client's retry backoff was now "250 ms".

来源

The prompt contract is short: write Markdown, never write a number, version, or URL directly — emit a token instead.

来源

claimcheck.py is stdlib‑only and runs in two stages. Stage one scans the raw draft; stage two scans the rendered page and confirms every surviving claim traces back to a registered value.

来源