Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

I Gave an LLM the Keys to Rewrite Its Own Prompt — Then Built a Gate That Said No 4,150 Times

Description and use

A safe prompt‑editing pipeline was built using a local Qwen 4B model and a deterministic six‑check gate, achieving zero false positives and catching all adversarial edits across four domains.

Problem

Building a self‑improving prompt optimizer risks unsafe edits; the gate must be deterministic and conservative to prevent drift and false promotions.

Outcome

Zero false positives, 5/5 adversarial edits caught, 12/12 Docker tests passed, 489/489 unit tests passed, 4,150 LLM calls at $0.00 cost.

Workflow

  • 1. Agent executes task and stores trace in SQLite
  • 2. Feedback Analyzer (LLM) reviews trace and proposes prompt edits
  • 3. A/B Test Engine runs candidate vs current prompt on a 40‑task set
  • 4. Promotion Gate applies six deterministic checks in order
  • 5. If all pass, promote; if near‑miss, log for human review; if reject, archive reasoning

Evidence

Gate false positive rate 0% (all runs, all domains)

Source

Sample floor — Minimum A/B trials completed. Default 5.

Source

4,150 LLM calls in the full field test: $0.00.

Source

  • Record date: 2026-09-07T04:07:34.035Z