I stopped fixing my prompt by hand. Open models fixed it while I napped

Описание и применение

Автоматический цикл улучшения промпта, использующий открытые модели и AcruxCore, повысил точность маршрутизации запросов банка с 37 % до 60 % на Gemma 3 4B и с 69 % до 74 % на Gemma 3 27B, без ручного редактирования.

Задача

Manually editing prompts for a bank message router was error‑prone and time‑consuming.

Результат

The prompt‑improvement loop, driven by open‑weight models and executed via AcruxCore’s MCP, eliminated manual edits and improved intent‑routing accuracy from 37% to 60% on Gemma 3 4B and from 69% to 74% on Gemma 3 27B, at a cost of $0.38.

Рабочий процесс

• Create lazy prompt (v1) and store via MCP
• Load 154 training examples with intent definitions
• Run optimizer to draft rewrites and judge scores
• Promote winning rewrite as new prompt version (v3)
• Evaluate on 308 held‑out messages

Доказательства

Fixing a prompt by hand is a loop I hate. I rewrite it, test it by hand, redeploy, and find that I fixed two cases and broke a third.

Источник

Gemma 3 4B 37% → 60% on 308 held‑out messages

Источник

The router never changes. AcruxCore stores each prompt as numbered versions, and an alias is a name that points at one version.

Источник

The optimizer wrote those definitions from the 154 training examples. On held‑out messages, wrong labels fell from 45% to 40%.

Источник