I tested my AI safety filter 400 times. It flagged my own innocent prompts 1 in 40
说明与用途
enki‑harness 测试工具发现 AI 安全过滤器的误报率为 1/40,促使删除错误的过滤步骤。
问题
AI safety filters often pass tests written by their creators, missing real false positives and negatives.
结果
The harness uncovered a 1‑in‑40 false‑positive rate on innocent prompts, leading to removal of a faulty moderation step and saving development time.
工作流
- 1. Install enki‑harness (MIT, plain Python).
- 2. Run harness locally or as MCP server to register filter with coding agents.
- 3. Execute hb judge to send cases to an open‑weight model (e.g., Llama Guard 3 8B via Ollama).
- 4. Harness generates twins, controls, rules‑off, attacker slice, and falsifier cases.
- 5. Collect flag rates and identify false positives.
- 6. Adjust or remove problematic filter rules based on results.
证据
enki‑harness builds the harness for you and adds the parts you'd skip: Twins, Controls, Rules off, Attacker slice, Falsifier
hb judge talks to open‑weight models running locally through Ollama (I used Llama Guard 3 8B). There are no API keys, no per-call cost, and nothing leaves the machine.
Run each one several times and the screen still flagged them at random, about 1 time in 40: a different sentence each time, never the same one twice in a row.
- 记录日期: 2026-10-22T21:11:38.048Z