I stopped fixing my prompt by hand. Open models fixed it while I napped

说明与用途

使用开源模型和 AcruxCore 的自动化提示改进循环，将银行消息路由器的准确率从 Gemma 3 4B 的 37% 提升到 60%，从 Gemma 3 27B 的 69% 提升到 74%，无需手动编辑。

问题

Manually editing prompts for a bank message router was error‑prone and time‑consuming.

结果

The prompt‑improvement loop, driven by open‑weight models and executed via AcruxCore’s MCP, eliminated manual edits and improved intent‑routing accuracy from 37% to 60% on Gemma 3 4B and from 69% to 74% on Gemma 3 27B, at a cost of $0.38.

工作流

• Create lazy prompt (v1) and store via MCP
• Load 154 training examples with intent definitions
• Run optimizer to draft rewrites and judge scores
• Promote winning rewrite as new prompt version (v3)
• Evaluate on 308 held‑out messages

证据

Fixing a prompt by hand is a loop I hate. I rewrite it, test it by hand, redeploy, and find that I fixed two cases and broke a third.

来源

Gemma 3 4B 37% → 60% on 308 held‑out messages

来源

The router never changes. AcruxCore stores each prompt as numbered versions, and an alias is a name that points at one version.

来源

The optimizer wrote those definitions from the 154 training examples. On held‑out messages, wrong labels fell from 45% to 40%.

来源