Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

tonesearch: find the angry minute in your call recordings, on your own machine

说明与用途

tonesearch 让小型呼叫团队能够在本地搜索录音中的内容和语调,而无需将音频上传到云端,使用开放权重模型 EmbeddingGemma 2 和 emotion2vec+ 进行隐私保护分析。

问题

Small call teams and agencies need to locate angry moments in call recordings without sending audio to external cloud services.

结果

tonesearch enables local, privacy‑preserving search for content and tone in call recordings, achieving high correlation with human irritation ratings (Spearman +0.66 when combining Gemma audio embeddings with emotion2vec). It runs at 18× real time on an M3 Mac with GPU and 3.4× on CPU.

工作流

  • Index call recordings by splitting on silence into 2‑10s segments and running EmbeddingGemma 2 and emotion2vec+ to produce embeddings and tone scores.
  • Store the embeddings and scores in a .npz file for fast retrieval.
  • Run a query by embedding the text with Gemma, optionally specifying a tone weight, and compute a combined score via cosine similarity and tone log‑odds.
  • Return the top N segments; clicking a row plays the audio segment; a separate tab shows the tone over the entire call with top spikes.
  • All processing occurs locally; no audio leaves the machine.

证据

It needs no transcript and runs on a laptop, so no audio leaves the machine.

来源

The demo data is HaessigDB, a public set of acted call-center speech (CC-BY-4.0, by Niklas Weller, Marc Christopher Grau and Ivo Blohm). These are actors, not real customers.

来源

On an M3 Mac, with Gemma on the GPU, indexing runs at 18x real time. On CPU only, it runs at 3.4x real time.

来源

The audio embedding scored the same as the transcript embedding. It hears the words about as well as a transcript does, and it adds almost nothing about tone.

来源

  • 记录日期: 2026-10-13T19:41:11.861Z