Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

I tested my AI safety filter 400 times. It flagged my own innocent prompts 1 in 40

说明与用途

enki‑harness 测试工具发现 AI 安全过滤器的误报率为 1/40,促使删除错误的过滤步骤。

问题

AI safety filters often pass tests written by their creators, missing real false positives and negatives.

结果

The harness uncovered a 1‑in‑40 false‑positive rate on innocent prompts, leading to removal of a faulty moderation step and saving development time.

工作流

  • 1. Install enki‑harness (MIT, plain Python).
  • 2. Run harness locally or as MCP server to register filter with coding agents.
  • 3. Execute hb judge to send cases to an open‑weight model (e.g., Llama Guard 3 8B via Ollama).
  • 4. Harness generates twins, controls, rules‑off, attacker slice, and falsifier cases.
  • 5. Collect flag rates and identify false positives.
  • 6. Adjust or remove problematic filter rules based on results.

证据

enki‑harness builds the harness for you and adds the parts you'd skip: Twins, Controls, Rules off, Attacker slice, Falsifier

来源

hb judge talks to open‑weight models running locally through Ollama (I used Llama Guard 3 8B). There are no API keys, no per-call cost, and nothing leaves the machine.

来源

Run each one several times and the screen still flagged them at random, about 1 time in 40: a different sentence each time, never the same one twice in a row.

来源

  • 记录日期: 2026-10-22T21:11:38.048Z