Teaching a Small Model to Recognise a Sport From Moving Dots

Description and use

Fine‑tuned Laya achieved 36 % accuracy on a 400‑clip sports‑recognition benchmark, showing sensitivity to snapshot order (8 % drop when shuffled).

Problem

Can a small language model learn to identify sports from sequences of player coordinates represented as moving dots, and does it capture temporal order?

Outcome

Fine‑tuned Laya achieved 36% accuracy on the 400‑clip benchmark, outperforming the official recipe and matching frontier models. It showed a clear dependence on snapshot order, losing 8 points when snapshots were shuffled, similar to MiniRocket and Opus 5.5.

Workflow

• Prepare 400 clips of 10 dots over 8 snapshots as text coordinates
• Split data into five folds grouped by game
• Fine‑tune Laya on each fold using 32 epochs, selecting the epoch with best held‑out validation
• Evaluate accuracy on test clips for ordered, shuffled, and single‑snapshot conditions
• Compare results to MiniRocket, DeepSets, and frontier language models

Evidence

Untouched, Laya gets 24% of the clips right, which is chance.

Source

With 32 epochs and early stopping, 0.36 in order against 0.27 shuffled and 0.27 with a single snapshot.

Source

The second row is the one I was after. Shuffling the snapshots costs Laya 8 points, of the same order as the 10 it cost Opus 5.5.

Source