Dark Matter Engine: A Sloth That Hunts for Missing Answers at 2 AM

Description and use

Dark Matter Engine automatically detects missing failure scenarios in documentation using a local Gemma model and text search, producing a report for manual review.

Problem

Documentation often omits failure scenarios, leaving gaps that can cause bugs such as double charging when a gateway response is lost.

Outcome

On a test set of nine spec pairs, the default check flagged 6 of 9 planted gaps and only 2 of 45 documented behaviors as false positives; a stricter mode flagged all 9 gaps but increased false positives to 23.

Workflow

• 1. Point the engine at a folder of Markdown or text docs.
• 2. Gemma 3 (4B) model proposes failure scenarios based on documented behaviors.
• 3. Plain lexical search retrieves evidence passages from the docs.
• 4. Gemma classifies each scenario as Answered, Partially answered, or Not established.
• 5. Python verifies that every quoted evidence exists in the retrieved text.
• 6. A second search with synonyms is performed for Not established cases.
• 7. The engine outputs an HTML report listing unresolved questions.
• 8. Run the tool on a schedule (e.g., 2 AM) to get daily reports.

Evidence

The Dark Matter Engine is an offline scan that reads your docs and lists the failure questions they never answer.

Source

On a real design document it found one genuine unanswered question and produced eight findings I had to throw out.

Source

Gemma proposes scenarios from sections the docs actually contain.

Source