Fine‑tuning a 7B model on 549 Chinese A‑share signal samples

Description and use

A small, imbalanced dataset of 549 Chinese A‑share signal samples limited the effectiveness of a 7B model fine‑tuned with LoRA. Adding capacity or epochs did not improve accuracy; the main bottleneck was dataset coverage and missing contextual features.

Problem

A small, imbalanced dataset of 549 samples across 12 stocks and 5 classes limited the effectiveness of a 7B model fine‑tuned with LoRA, causing the model to misclassify rare but critical signals.

Outcome

The experiment showed that adding model capacity or training epochs did not improve overall accuracy; the primary limitation was dataset imbalance and missing contextual features. The author released a free, key‑less data pipeline, regime‑split backtest, and detailed failure log to aid reproducibility.

Workflow

• Collect daily snapshots with a cron job using Tencent’s free quote endpoint
• Parse GBK‑encoded data into price, volume, etc.
• Create chat‑style training rows (instruction/input/output) with structured verdicts
• Fine‑tune Qwen2.5‑7B‑Instruct with QLoRA (rank 16, 3 epochs) on an RTX 3080
• Run evaluation with fixed test set, adjust rank and epochs, observe no score improvement
• Analyze errors: rare class collapses into common class due to 5.8:1 prior
• Identify infrastructure bugs (trl version, custom tokens, environment isolation)
• Plan improvements: balance classes, add regime context, time‑based split, freeze eval set, route low‑confidence outputs for re‑check
• Publish data pipeline, backtest, and failure log on GitHub
• Offer packaged skill for agents

Evidence

The gap between the most and least frequent class is 5.8:1. The rarest class — the one that tells you to get out — is also the one that matters most.

Source

First run (rank 16, 3 epochs): the model produced fluent Chinese analysis but didn't reliably follow the output format of the training rows.

Source

When your dataset is coverage‑bound, adding capacity only reorders your errors. It doesn't create knowledge that isn't in the examples.

Source