I Gave an LLM the Keys to Rewrite Its Own Prompt — Then Built a Gate That Said No 4,150 Times
说明与用途
使用本地 Qwen 4B 模型和六项确定性检查的门,构建了安全的提示自我改进管道,实现零误报并捕获所有对抗性编辑。
问题
Building a self‑improving prompt optimizer risks unsafe edits; the gate must be deterministic and conservative to prevent drift and false promotions.
结果
Zero false positives, 5/5 adversarial edits caught, 12/12 Docker tests passed, 489/489 unit tests passed, 4,150 LLM calls at $0.00 cost.
工作流
- 1. Agent executes task and stores trace in SQLite
- 2. Feedback Analyzer (LLM) reviews trace and proposes prompt edits
- 3. A/B Test Engine runs candidate vs current prompt on a 40‑task set
- 4. Promotion Gate applies six deterministic checks in order
- 5. If all pass, promote; if near‑miss, log for human review; if reject, archive reasoning
证据
Gate false positive rate 0% (all runs, all domains)
Sample floor — Minimum A/B trials completed. Default 5.
4,150 LLM calls in the full field test: $0.00.
- 记录日期: 2026-09-07T04:07:34.035Z