I Tried 4 Models to Save My Self-Improving Agent. All 4 Failed.

Описание и применение

Все четыре модели LLM не смогли улучшить собственные подсказки из‑за локальной оптимизации поисковой стратегии. Пайплайн доказан, но узкое место – отсутствие генерации разнообразных кандидатов.

Задача

The self‑editing agent could not produce promotable edits because its search strategy stuck in local optima, regardless of model size.

Результат

All four tested models failed to produce promotable edits; the issue was the search strategy, not model size. The agent’s pipeline was proven, but the bottleneck was identified and a concrete fix path defined.

Рабочий процесс

• 1. Generate one edit proposal from the analyzer. 2. Run A/B test against a failure batch. 3. If the edit passes the gate, promote; otherwise repeat. 4. Repeat until a promotable edit is found or resources exhausted.
• 5. In v0.3.0, modify step 1 to generate 10 diverse candidates. 6. Apply cheap scoring on the failure batch. 7. Select the best candidate for the A/B test. 8. Iterate until promotion or timeout.

Доказательства

None of them did.

Источник

The search strategy was.

Источник

The analyzer generates one proposal, sees the same failures next iteration, and proposes a slightly different version of the same failed edit.

Источник