I Tried 4 Models to Save My Self-Improving Agent. All 4 Failed.

说明与用途

所有四个 LLM 模型因搜索策略陷入局部最优而未能改进自身提示。管道已验证，但瓶颈是缺乏多样化候选生成。

问题

The self‑editing agent could not produce promotable edits because its search strategy stuck in local optima, regardless of model size.

结果

All four tested models failed to produce promotable edits; the issue was the search strategy, not model size. The agent’s pipeline was proven, but the bottleneck was identified and a concrete fix path defined.

工作流

• 1. Generate one edit proposal from the analyzer. 2. Run A/B test against a failure batch. 3. If the edit passes the gate, promote; otherwise repeat. 4. Repeat until a promotable edit is found or resources exhausted.
• 5. In v0.3.0, modify step 1 to generate 10 diverse candidates. 6. Apply cheap scoring on the failure batch. 7. Select the best candidate for the A/B test. 8. Iterate until promotion or timeout.

证据

None of them did.

来源

The search strategy was.

来源

The analyzer generates one proposal, sees the same failures next iteration, and proposes a slightly different version of the same failed edit.

来源