Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

tonesearch: find the angry minute in your call recordings, on your own machine

Описание и применение

tonesearch позволяет небольшим агентствам локально находить эмоциональные моменты в звонках, не отправляя аудио в облако, используя открытые модели EmbeddingGemma 2 и emotion2vec+.

Задача

Small call teams and agencies need to locate angry moments in call recordings without sending audio to external cloud services.

Результат

tonesearch enables local, privacy‑preserving search for content and tone in call recordings, achieving high correlation with human irritation ratings (Spearman +0.66 when combining Gemma audio embeddings with emotion2vec). It runs at 18× real time on an M3 Mac with GPU and 3.4× on CPU.

Рабочий процесс

  • Index call recordings by splitting on silence into 2‑10s segments and running EmbeddingGemma 2 and emotion2vec+ to produce embeddings and tone scores.
  • Store the embeddings and scores in a .npz file for fast retrieval.
  • Run a query by embedding the text with Gemma, optionally specifying a tone weight, and compute a combined score via cosine similarity and tone log‑odds.
  • Return the top N segments; clicking a row plays the audio segment; a separate tab shows the tone over the entire call with top spikes.
  • All processing occurs locally; no audio leaves the machine.

Доказательства

It needs no transcript and runs on a laptop, so no audio leaves the machine.

Источник

The demo data is HaessigDB, a public set of acted call-center speech (CC-BY-4.0, by Niklas Weller, Marc Christopher Grau and Ivo Blohm). These are actors, not real customers.

Источник

On an M3 Mac, with Gemma on the GPU, indexing runs at 18x real time. On CPU only, it runs at 3.4x real time.

Источник

The audio embedding scored the same as the transcript embedding. It hears the words about as well as a transcript does, and it adds almost nothing about tone.

Источник

  • Дата записи: 2026-10-13T19:41:11.861Z