LazyDecider: I taught an open-weight agent to fine-tune a model while I slept

说明与用途

使用开源 Gemma 4 E4B 代理和 ASHA 剪枝的自动化夜间管道，消除了手动超参数调优，生成 0.8B 决策模型，在封闭测试集上实现 0.770 准确率和 0.094 KL‑from‑gold。

问题

Automating hyperparameter tuning so the engineer never babysits a sweep again

结果

The automated pipeline produced a 0.8B decision model with 0.770 accuracy and 0.094 KL‑from‑gold on a sealed test, outperforming the zero‑shot baseline and confirming transfer to a 4B model

工作流

• 1. Run Gemma 4 E4B locally to propose a hyperparameter config with a one‑line reason
• 2. ASHA pruner trains the config and kills it early if it loses
• 3. Keep the run only if it beats the current best on a frozen split, otherwise revert
• 4. In the morning retrain the winning config once
• 5. Score on a sealed test set
• 6. Run a Tinker confirmation on a larger model

证据

I will happily spend a week automating a job that takes ten minutes, as long as I never have to do the ten minutes again.

来源

A local, open-weight agent (Gemma 4 E4B) reads the experiment log, proposes one training config with a one-line reason, an ASHA pruner trains it and kills it early if it's losing, and the run is kept only if it beats the current best on a frozen split — otherwise reverted.

来源

The result: a 0.8B decision model found autonomously that lands 0.770 accuracy and a very low KL-from-gold of 0.094 on a sealed test

来源