Self-Supervised Temporal Pattern Mining for coastal climate resilience planning for extreme data sparsity scenarios

Описание и применение

Построена система самосовершенствования для анализа временных паттернов в условиях экстремальной нехватки данных, позволяющая реконструировать пропущенные физические сигналы и улучшать прогнозы штормовых приливов.

Задача

Coastal resilience projects in developing regions lack continuous in‑situ sensor data, leading to sparse, noisy time‑series that hinder supervised forecasting models.

Результат

The self‑supervised pipeline learns meaningful climate signals from sparse, noisy data, enabling reconstruction of missing physics and improving forecasting for coastal resilience in data‑scarce regions.

Рабочий процесс

• 1. Treat sparse time‑series as a partially observed dynamical system. 2. Use Contrastive Predictive Coding (CPC) with InfoNCE loss to learn representations that distinguish true future states from negative samples. 3. Adapt Masked Autoencoders (MAE) to spatiotemporal climate data by masking entire time steps. 4. Combine a Temporal Convolutional Network (TCN) for local feature extraction with a Transformer for long‑range dependencies. 5. Implement Time‑Aware Positional Encoding using a Gaussian kernel to handle irregular timestamps. 6. Simulate sparsity with block masking (contiguous months) to train the model on realistic gaps. 7. Train the Masked Temporal Autoencoder on satellite imagery and sparse buoy data. 8. Evaluate reconstruction quality and downstream forecasting performance.

Доказательства

Traditional Recurrent Neural Networks (RNNs) and even Transformers struggle here. They tend to overfit to the noise or fail to converge due to the lack of continuous sequences.

Источник

In my experimentation with CPC, I realized that the "InfoNCE" loss function is particularly powerful for sparse data.

Источник

While learning about Vision Transformers, I observed that the MAE architecture (masking 75% of patches) could be adapted for spatiotemporal climate data.

Источник

To train the self-supervised model, I artificially created sparsity. During my investigation of data augmentation for climate data, I found that block masking (masking contiguous months) was more effective than random masking for learning seasonal patterns.

Источник