Generative Simulation Benchmarking for planetary geology survey missions for extreme data sparsity scenarios

Описание и применение

Генеративное моделирование позволяет оценивать модели машинного обучения для геологических исследований планет, где данные крайне редки, избегая утечки информации из тренировочного набора.

Задача

Evaluating machine learning models for planetary geology with extremely sparse, biased, and geographically narrow data leads to over‑optimistic benchmarks that fail in new terrains.

Результат

Using generative benchmarks provides unlimited labeled data from distributions unseen by the model, enabling a realistic assessment of generalization in extreme data sparsity scenarios.

Рабочий процесс

• Build a generative terrain model conditioned on geological parameters
• Generate synthetic scenes with known ground truth
• Sample a stratified grid of geological conditions
• Evaluate the perception model across the grid and report robustness profiles

Доказательства

The benchmark had leaked information. My validation set shared the same simulated geological province as my training set, and my model had simply memorized the distribution of that one region.

Источник

Because the generator knows the ground truth by construction, you get unlimited labeled data from distributions your model has never seen.

Источник