I Built a Local AI Music Studio

Description and use

Miso is a local AI music studio that unifies multiple music models and post‑processing tools into a single web interface, streamlining generation, editing, and reproducibility.

Problem

Users want an interface to experiment with local music models and manage post‑generation tasks like editing, stem separation, voice conversion, and MIDI transcription without juggling command‑line tools.

Outcome

Miso enables local, GPU‑accelerated music generation and post‑processing workflows, reducing the need for multiple tools and improving reproducibility.

Workflow

• 1. User selects a model and inputs a prompt via the guided builder. 2. Miso sends the request to the audio.cpp backend. 3. Generated audio is stored as a take with metadata. 4. User can repaint sections, split stems, or apply voice conversion. 5. Stems can be mixed, exported, or used as input for other models. 6. Audio can be transcribed to MIDI, converted to an ABC score, and fed to YuE2 for new arrangements. 7. All actions are recorded in the project history for reproducibility.

Evidence

Miso is a self-hosted local music generation and remix studio built on top of audio.cpp.

Source

The whole thing runs as two Docker containers. One runs Miso itself. The other runs audio.cpp and owns the GPU models.

Source

Miso can also split a track into stems. Once the stems exist, they get their own mixer.

Source

The current release, 1.4.0, connects MIDI transcription to YuE2's score support.

Source