Self-Supervised Temporal Pattern Mining for coastal climate resilience planning for extreme data sparsity scenarios

Description and use

A self‑supervised pipeline was developed to mine temporal patterns from sparse coastal climate data, enabling reconstruction of missing physics and improved storm surge forecasting.

Problem

Coastal resilience projects in developing regions lack continuous in‑situ sensor data, leading to sparse, noisy time‑series that hinder supervised forecasting models.

Outcome

The self‑supervised pipeline learns meaningful climate signals from sparse, noisy data, enabling reconstruction of missing physics and improving forecasting for coastal resilience in data‑scarce regions.

Workflow

• 1. Treat sparse time‑series as a partially observed dynamical system. 2. Use Contrastive Predictive Coding (CPC) with InfoNCE loss to learn representations that distinguish true future states from negative samples. 3. Adapt Masked Autoencoders (MAE) to spatiotemporal climate data by masking entire time steps. 4. Combine a Temporal Convolutional Network (TCN) for local feature extraction with a Transformer for long‑range dependencies. 5. Implement Time‑Aware Positional Encoding using a Gaussian kernel to handle irregular timestamps. 6. Simulate sparsity with block masking (contiguous months) to train the model on realistic gaps. 7. Train the Masked Temporal Autoencoder on satellite imagery and sparse buoy data. 8. Evaluate reconstruction quality and downstream forecasting performance.

Evidence

Traditional Recurrent Neural Networks (RNNs) and even Transformers struggle here. They tend to overfit to the noise or fail to converge due to the lack of continuous sequences.

Source

In my experimentation with CPC, I realized that the "InfoNCE" loss function is particularly powerful for sparse data.

Source

While learning about Vision Transformers, I observed that the MAE architecture (masking 75% of patches) could be adapted for spatiotemporal climate data.

Source

To train the self-supervised model, I artificially created sparsity. During my investigation of data augmentation for climate data, I found that block masking (masking contiguous months) was more effective than random masking for learning seasonal patterns.

Source