Dark Matter Engine: A Sloth That Hunts for Missing Answers at 2 AM

Описание и применение

Инструмент Dark Matter Engine автоматически выявляет пропущенные сценарии отказа в документации, используя локальную модель Gemma и поиск по тексту, и генерирует отчёт для ручного обзора.

Задача

Documentation often omits failure scenarios, leaving gaps that can cause bugs such as double charging when a gateway response is lost.

Результат

On a test set of nine spec pairs, the default check flagged 6 of 9 planted gaps and only 2 of 45 documented behaviors as false positives; a stricter mode flagged all 9 gaps but increased false positives to 23.

Рабочий процесс

• 1. Point the engine at a folder of Markdown or text docs.
• 2. Gemma 3 (4B) model proposes failure scenarios based on documented behaviors.
• 3. Plain lexical search retrieves evidence passages from the docs.
• 4. Gemma classifies each scenario as Answered, Partially answered, or Not established.
• 5. Python verifies that every quoted evidence exists in the retrieved text.
• 6. A second search with synonyms is performed for Not established cases.
• 7. The engine outputs an HTML report listing unresolved questions.
• 8. Run the tool on a schedule (e.g., 2 AM) to get daily reports.

Доказательства

The Dark Matter Engine is an offline scan that reads your docs and lists the failure questions they never answer.

Источник

On a real design document it found one genuine unanswered question and produced eight findings I had to throw out.

Источник

Gemma proposes scenarios from sections the docs actually contain.

Источник