Fine‑tuning a 7B model on 549 Chinese A‑share signal samples

Описание и применение

Маленький, несбалансированный набор из 549 примеров ограничил эффективность 7B модели, обученной с LoRA. Добавление параметров и эпох не улучшило точность; ключевой проблемой была нехватка контекста и дисбаланс классов.

Задача

A small, imbalanced dataset of 549 samples across 12 stocks and 5 classes limited the effectiveness of a 7B model fine‑tuned with LoRA, causing the model to misclassify rare but critical signals.

Результат

The experiment showed that adding model capacity or training epochs did not improve overall accuracy; the primary limitation was dataset imbalance and missing contextual features. The author released a free, key‑less data pipeline, regime‑split backtest, and detailed failure log to aid reproducibility.

Рабочий процесс

• Collect daily snapshots with a cron job using Tencent’s free quote endpoint
• Parse GBK‑encoded data into price, volume, etc.
• Create chat‑style training rows (instruction/input/output) with structured verdicts
• Fine‑tune Qwen2.5‑7B‑Instruct with QLoRA (rank 16, 3 epochs) on an RTX 3080
• Run evaluation with fixed test set, adjust rank and epochs, observe no score improvement
• Analyze errors: rare class collapses into common class due to 5.8:1 prior
• Identify infrastructure bugs (trl version, custom tokens, environment isolation)
• Plan improvements: balance classes, add regime context, time‑based split, freeze eval set, route low‑confidence outputs for re‑check
• Publish data pipeline, backtest, and failure log on GitHub
• Offer packaged skill for agents

Доказательства

The gap between the most and least frequent class is 5.8:1. The rarest class — the one that tells you to get out — is also the one that matters most.

Источник

First run (rank 16, 3 epochs): the model produced fluent Chinese analysis but didn't reliably follow the output format of the training rows.

Источник

When your dataset is coverage‑bound, adding capacity only reorders your errors. It doesn't create knowledge that isn't in the examples.

Источник