Nimbus Choir: Point Your Phone at the Sky, Hear It Sing

说明与用途

Nimbus Choir 是一款完全离线的移动应用，将实时云图像转化为环境音乐，保护隐私并消除网络依赖。

问题

Providing an offline, privacy‑preserving ambient audio experience that turns real‑time sky imagery into generative music without relying on cloud services.

结果

Nimbus Choir delivers a fully offline, privacy‑respecting ambient audio experience that maps real‑time cloud imagery to sound, with optional online Earth mode for satellite data.

工作流

• 1. Point phone camera at sky
• 2. Camera pipeline segments sky and detects clouds
• 3. IoU tracker follows cloud movement
• 4. DINOv2‑based classifier assigns cloud type, size, height, drift
• 5. TabPFN forecaster predicts tension from cloud cover and pressure trend
• 6. Web Audio engine generates ambient sound based on cloud attributes
• 7. Optional Earth mode fetches live NASA satellite tiles via proxy if online

证据

Nimbus Choir points your phone's camera at the sky and turns it into a soft, generative ambient piece -- entirely offline, no account, no cloud service watching what's above you.

来源

The sound engine is hand‑built Web Audio: detuned oscillator pads through a lowpass + slow tremolo, plus physically‑modeled partial‑based chimes.

来源

The classifier pipeline. Public CCSN cloud dataset → dedupe/augment → DINOv2 (open‑weight, frozen) + trainable head, distilled with optional Tinker teacher soft‑labels for extra accuracy, but the version actually shipped here trained on ground truth only -- $0 spent, no closed model touched at all.

来源

The narrator originally used ElevenLabs for voice. It worked, but it meant: no network → no voice, an API key to manage, and a per‑character cost for something meant to run in a field with no signal. I swapped it for Piper, an open‑weight local TTS.

来源

Runs with no internet. The main camera mode -- segmentation, tracking, classification, the entire sound engine -- never makes a network call after first load.

来源