Reading a mobile game screen pixel by pixel

说明与用途

为《部落冲突》屏幕创建了420万像素级掩码数据集，训练出能够准确检测禁止部署区域的分割模型，验证集IoU达0.996。

问题

Clash of Clans players need to identify loot, troop placement, and deploy zones within a 30‑second window, but existing datasets lack pixel‑level annotations for the game’s complex UI and overlapping objects.

结果

The deploy zone model achieved 0.996 IoU on the validation set, accurately delineating the no‑deploy area and preventing mask leakage onto the army bar.

工作流

• 1. Capture screenshots at three stages: scout screen, during battle, end of battle.
• 2. Annotate each visible object as a polygon mask, accounting for overlapping and partial visibility.
• 3. Compile 6,909 frames into 4.2M masks, split validation set by base to avoid memorization.
• 4. Train a U‑Net segmentation model (MobileNetV2 or EfficientNet‑B4) to predict the no‑deploy area mask.
• 5. Evaluate on validation set, achieving 0.996 IoU for the no‑deploy area.

证据

I built one.

来源

6,909 frames, around 4.2M masks. YOLO seg and COCO.

来源

0.996 IoU on the no‑deploy area on val.

来源