How I built a riff vault for Sam with open-source AI

Description and use

RiffSalad lets a guitarist quickly locate recordings by describing their musical traits, eliminating the need to sift through non-descriptive filenames.

Problem

Sam had hundreds of audio files with non-descriptive names and could not locate a riff based on its musical characteristics, such as key, tempo, or feel.

Outcome

RiffSalad enables Sam to locate riffs by describing their musical attributes, eliminating reliance on filenames and reducing time spent scrolling through files.

Workflow

• Import WAV/MP3/M4A files into RiffSalad
• Librosa analyzes audio for BPM, key, energy, note density, duration
• Basic Pitch ONNX model extracts MIDI
• Ollama LLM (qwen2.5 1.5B) parses search query into structured filters
• Plain code prunes results based on measurable facts
• Embedding similarity (nomic-embed-text via Ollama) re-ranks remaining riffs
• Optional voice note transcription with faster-whisper tiny model
• User searches via natural language, receives ranked list of riffs
• User can edit tags, view waveform, and find similar riffs

Evidence

RiffSalad, a local AI vault, automatically extracts BPM, key, energy, note density, and MIDI transcription, generates descriptive tags, and allows natural language search such as "heavy drop C riff around 140 BPM"

Source

The LLM turns your sentence into structured filters: BPM range, key, energy, mood words

Source

On an M4 mac air, a riff takes less than 15 seconds to analyze.

Source

Sam still has 103 files called recording-something. He just doesn't need the filenames anymore.

Source