I Trained an Open Decision Model on a Free GPU in 46 Minutes. On My Tests, It Beat Laya.

Описание и применение

Типичный модель, обученная за 46 минут на бесплатном GPU, превзошла Laya по точности и скорости на тестах, не требуя генерации текста и последующего парсинга.

Задача

Large language models are slow, costly, and produce text that must be parsed for decision tasks.

Результат

Typic achieved 69.9% accuracy on unseen tasks, outperforming Laya’s 56.7% and matching its intent routing accuracy, while running in 29 ms per question versus Laya’s 37 ms.

Рабочий процесс

• Collect 25 public datasets and convert each example to (context, question, options, answer) format.
• Apply extensive augmentation: vary question wording, label wording, option order, include "none of these", flip yes/no questions.
• Train Ettin‑400M encoder with a small head, using one T4 GPU, batch size 8, gradient accumulation, and checkpointing for 46 minutes.
• Fit a temperature parameter for calibration.
• Evaluate on 2,000 unseen questions across Banking77, CommonsenseQA, COPA, OpenBookQA, and long‑context routing tests.
• Compare results with Laya’s official package.
• Publish the model on Hugging Face.

Доказательства

I trained three sizes along the way. On questions from tasks the model never saw during training: The tiny 68M pilot got only 33% on a five‑option test (random is 20%). 150M reached 58.3% across four unseen tasks. 400M reached 69.9%.

Источник

Typic was 13 points more accurate overall, tied Laya on intent routing, and answered in 29 ms vs. 37 ms per question.

Источник

I placed 20 support requests after 0 to 7,000 tokens of filler… Typic kept 17 or 18 out of 20 correct up to 7,000 tokens. Both Laya checkpoints dropped to 5 or 6 out of 20.

Источник