Show Dev: Your Face Is Now a Music Generator

说明与用途

实时音乐生成器，基于用户情绪实时生成音乐，使用客户端面部检测和服务器端音频合成。

问题

Providing a continuous, real‑time music experience that reflects a user’s current emotional state without sending webcam data to a server.

结果

A real‑time, privacy‑preserving music generator that reacts to facial emotions, avoiding serverless cold starts and maintaining stateful audio transitions.

工作流

• 1. Capture webcam frames in the browser.
• 2. Run face‑landmark detection client‑side to extract emotion probabilities.
• 3. Debounce updates (250 ms) and POST emotion vector to /api/mood.
• 4. Server receives vector, maps it to musical parameters (tempo, key, layers).
• 5. Server streams parameter updates to the client.
• 6. Client updates Web Audio API mixer with new parameters.
• 7. Music plays continuously, adapting to real‑time emotional changes.

证据

Face detection runs entirely client‑side. Webcam frames → face landmarks → a small vector of emotion probabilities. None of that leaves your machine.

来源

The music generation, though, lives in its own Railway service. That was deliberate.

来源