How I built a riff vault for Sam with open-source AI

Описание и применение

RiffSalad позволяет гитаристу быстро находить записи, описывая их музыкальные характеристики, без необходимости просматривать файлы по именам.

Задача

Sam had hundreds of audio files with non-descriptive names and could not locate a riff based on its musical characteristics, such as key, tempo, or feel.

Результат

RiffSalad enables Sam to locate riffs by describing their musical attributes, eliminating reliance on filenames and reducing time spent scrolling through files.

Рабочий процесс

• Import WAV/MP3/M4A files into RiffSalad
• Librosa analyzes audio for BPM, key, energy, note density, duration
• Basic Pitch ONNX model extracts MIDI
• Ollama LLM (qwen2.5 1.5B) parses search query into structured filters
• Plain code prunes results based on measurable facts
• Embedding similarity (nomic-embed-text via Ollama) re-ranks remaining riffs
• Optional voice note transcription with faster-whisper tiny model
• User searches via natural language, receives ranked list of riffs
• User can edit tags, view waveform, and find similar riffs

Доказательства

RiffSalad, a local AI vault, automatically extracts BPM, key, energy, note density, and MIDI transcription, generates descriptive tags, and allows natural language search such as "heavy drop C riff around 140 BPM"

Источник

The LLM turns your sentence into structured filters: BPM range, key, energy, mood words

Источник

On an M4 mac air, a riff takes less than 15 seconds to analyze.

Источник

Sam still has 103 files called recording-something. He just doesn't need the filenames anymore.

Источник