Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

What the DOE GEMS Prize metric actually rewards (and a free local scorer)

说明与用途

DOE GEMS 指标更重视召回率,提供闭式阈值。正确处理已知断层掩模并延伸断层尖端可提升竞赛成绩。

问题

Determine how to optimize submissions for the DOE GEMS Prize by understanding the metric and adjusting model thresholds and handling of known faults.

结果

The local scorer and baseline reveal that maximizing recall, using a closed‑form threshold, and properly handling known faults can significantly improve competition scores.

工作流

  • Read the metric formula TP/(TP+0.2FP+0.8FN).
  • Derive that recall is worth four times precision.
  • Use Dinkelbach’s trick to compute a closed‑form threshold: mark pixels with fault probability >0.2×lambda for target score lambda.
  • Adjust the threshold (e.g., at lambda=0.4 use 0.08 instead of 0.5).
  • Handle known faults: suppress halo false positives and optionally extend fault tips beyond mapped tips.
  • Implement the scorer and baseline in Python (MIT licensed).
  • Run offline tests (27) and prepare GeoTIFF submission checker.
  • Submit to DrivenData and monitor leaderboard for extension length effects.

证据

Recall is worth four times precision.

来源

If your model can reach a score of lambda, a pixel is worth marking when its fault probability exceeds 0.2 x lambda.

来源

The organisers confirmed on the forum that USGS/INGENIOUS fault pixels are excluded from scoring, but the pixels next to them are scored normally against the new faults.

来源

  • 记录日期: 2026-10-14T04:41:36.652Z