Nimbus Choir: Point Your Phone at the Sky, Hear It Sing

Описание и применение

Nimbus Choir – автономное приложение, превращающее облака в музыку, работающее полностью офлайн и не передавая данные в облако.

Задача

Providing an offline, privacy‑preserving ambient audio experience that turns real‑time sky imagery into generative music without relying on cloud services.

Результат

Nimbus Choir delivers a fully offline, privacy‑respecting ambient audio experience that maps real‑time cloud imagery to sound, with optional online Earth mode for satellite data.

Рабочий процесс

• 1. Point phone camera at sky
• 2. Camera pipeline segments sky and detects clouds
• 3. IoU tracker follows cloud movement
• 4. DINOv2‑based classifier assigns cloud type, size, height, drift
• 5. TabPFN forecaster predicts tension from cloud cover and pressure trend
• 6. Web Audio engine generates ambient sound based on cloud attributes
• 7. Optional Earth mode fetches live NASA satellite tiles via proxy if online

Доказательства

Nimbus Choir points your phone's camera at the sky and turns it into a soft, generative ambient piece -- entirely offline, no account, no cloud service watching what's above you.

Источник

The sound engine is hand‑built Web Audio: detuned oscillator pads through a lowpass + slow tremolo, plus physically‑modeled partial‑based chimes.

Источник

The classifier pipeline. Public CCSN cloud dataset → dedupe/augment → DINOv2 (open‑weight, frozen) + trainable head, distilled with optional Tinker teacher soft‑labels for extra accuracy, but the version actually shipped here trained on ground truth only -- $0 spent, no closed model touched at all.

Источник

The narrator originally used ElevenLabs for voice. It worked, but it meant: no network → no voice, an API key to manage, and a per‑character cost for something meant to run in a field with no signal. I swapped it for Piper, an open‑weight local TTS.

Источник

Runs with no internet. The main camera mode -- segmentation, tracking, classification, the entire sound engine -- never makes a network call after first load.

Источник