Nimbus Choir: Point Your Phone at the Sky, Hear It Sing

Description and use

Nimbus Choir is an offline mobile app that sonifies real‑time cloud imagery into ambient music, preserving privacy and eliminating network dependencies.

Problem

Providing an offline, privacy‑preserving ambient audio experience that turns real‑time sky imagery into generative music without relying on cloud services.

Outcome

Nimbus Choir delivers a fully offline, privacy‑respecting ambient audio experience that maps real‑time cloud imagery to sound, with optional online Earth mode for satellite data.

Workflow

• 1. Point phone camera at sky
• 2. Camera pipeline segments sky and detects clouds
• 3. IoU tracker follows cloud movement
• 4. DINOv2‑based classifier assigns cloud type, size, height, drift
• 5. TabPFN forecaster predicts tension from cloud cover and pressure trend
• 6. Web Audio engine generates ambient sound based on cloud attributes
• 7. Optional Earth mode fetches live NASA satellite tiles via proxy if online

Evidence

Nimbus Choir points your phone's camera at the sky and turns it into a soft, generative ambient piece -- entirely offline, no account, no cloud service watching what's above you.

Source

The sound engine is hand‑built Web Audio: detuned oscillator pads through a lowpass + slow tremolo, plus physically‑modeled partial‑based chimes.

Source

The classifier pipeline. Public CCSN cloud dataset → dedupe/augment → DINOv2 (open‑weight, frozen) + trainable head, distilled with optional Tinker teacher soft‑labels for extra accuracy, but the version actually shipped here trained on ground truth only -- $0 spent, no closed model touched at all.

Source

The narrator originally used ElevenLabs for voice. It worked, but it meant: no network → no voice, an API key to manage, and a per‑character cost for something meant to run in a field with no signal. I swapped it for Piper, an open‑weight local TTS.

Source

Runs with no internet. The main camera mode -- segmentation, tracking, classification, the entire sound engine -- never makes a network call after first load.

Source