Make the Draft Refuse: An Abstention Ledger for Model-Written Docs

说明与用途

在此工作流程中，LLM 在生成文档时插入标记以标识无法验证的声明。解析器提取这些标记并检测隐藏的断言，生成路由表供审阅者验证和纠正内容。

问题

Model‑written docs often contain silent failures such as incorrect version pins or rate limits that reviewers miss because they look like fluent prose.

结果

The workflow turns invisible risks into countable objects, enabling systematic review and reducing silent failures in documentation.

工作流

• 1. Define a stable marker in the system prompt: <!-- NEEDS id="<short-slug>" owner="<team-or-person>" question="<one question>" -->
• 2. Run the LLM to draft prose, structure, and links, allowing it to emit markers when it cannot cite a fact.
• 3. Parse the draft with a Python script that extracts markers and detects silent assertions (versions, commands, URLs, quantities).
• 4. Generate a routing table mapping each marker to its line number, owner, and question.
• 5. Reviewers act on the table, filling in facts or correcting assertions.
• 6. Re‑run the draft if necessary to eliminate remaining markers.

证据

Instead of asking the model to be cautious in prose, you give it a syntactic obligation: when a claim depends on a fact it cannot cite from your repository, it must emit a marker and stop.

来源

The parser does two jobs: it extracts every marker, and it finds silent assertions — version numbers, shell commands, URLs, and unit-bearing figures that appear without a marker.

来源