LazyDecider: I taught an open-weight agent to fine-tune a model while I slept

Описание и применение

Автоматизированный ночной пайплайн, использующий открытый агент Gemma 4 E4B и ASHA‑принтер, позволяет без ручного участия подбирать гиперпараметры модели 0.8B, достигая 0.770 точности и 0.094 KL‑from‑gold на запечатанном тесте.

Задача

Automating hyperparameter tuning so the engineer never babysits a sweep again

Результат

The automated pipeline produced a 0.8B decision model with 0.770 accuracy and 0.094 KL‑from‑gold on a sealed test, outperforming the zero‑shot baseline and confirming transfer to a 4B model

Рабочий процесс

• 1. Run Gemma 4 E4B locally to propose a hyperparameter config with a one‑line reason
• 2. ASHA pruner trains the config and kills it early if it loses
• 3. Keep the run only if it beats the current best on a frozen split, otherwise revert
• 4. In the morning retrain the winning config once
• 5. Score on a sealed test set
• 6. Run a Tinker confirmation on a larger model

Доказательства

I will happily spend a week automating a job that takes ten minutes, as long as I never have to do the ten minutes again.

Источник

A local, open-weight agent (Gemma 4 E4B) reads the experiment log, proposes one training config with a one-line reason, an ASHA pruner trains it and kills it early if it's losing, and the run is kept only if it beats the current best on a frozen split — otherwise reverted.

Источник

The result: a 0.8B decision model found autonomously that lands 0.770 accuracy and a very low KL-from-gold of 0.094 on a sealed test

Источник