Dark Matter Engine: A Sloth That Hunts for Missing Answers at 2 AM

说明与用途

Dark Matter Engine 使用本地 Gemma 模型和文本搜索自动检测文档中缺失的失败场景，并生成供人工审核的报告。

问题

Documentation often omits failure scenarios, leaving gaps that can cause bugs such as double charging when a gateway response is lost.

结果

On a test set of nine spec pairs, the default check flagged 6 of 9 planted gaps and only 2 of 45 documented behaviors as false positives; a stricter mode flagged all 9 gaps but increased false positives to 23.

工作流

• 1. Point the engine at a folder of Markdown or text docs.
• 2. Gemma 3 (4B) model proposes failure scenarios based on documented behaviors.
• 3. Plain lexical search retrieves evidence passages from the docs.
• 4. Gemma classifies each scenario as Answered, Partially answered, or Not established.
• 5. Python verifies that every quoted evidence exists in the retrieved text.
• 6. A second search with synonyms is performed for Not established cases.
• 7. The engine outputs an HTML report listing unresolved questions.
• 8. Run the tool on a schedule (e.g., 2 AM) to get daily reports.

证据

The Dark Matter Engine is an offline scan that reads your docs and lists the failure questions they never answer.

来源

On a real design document it found one genuine unanswered question and produced eight findings I had to throw out.

来源

Gemma proposes scenarios from sections the docs actually contain.

来源