My speech-flaw detector flagged 41 false alarms a minute. One line of math fixed it.

Описание и применение

Детектор дефектов речи Podium изначально выдавал 41 ложноположительный сигнал в минуту на чистой записи другого диктора. После разделения стиля и дефектов и добавления признака голосового интервала ложные тревоги снизились до 3 в минуту, а точность F1 выросла до 0,51.

Задача

A speech‑flaw detector flagged 41 false alarms per minute on a clean recording from a different speaker.

Результат

False alarms on clean different‑speaker recordings reduced from 41/min to 3/min; F1 on held‑out set reached 0.51 with mean IoU 0.81.

Рабочий процесс

• 1. Force‑align both recordings to the same transcript using torchaudio’s MMS_FA.
• 2. Compute speaker‑normalised features (pitch, loudness, articulation rate, pause, voiced‑gap, high‑frequency energy).
• 3. Build a labelled dataset by injecting synthetic flaws into recordings and preserving exact timestamps.
• 4. Score each word with two robust z‑scores: deviation from reference (z_ref) and deviation from the user’s own style (z_self).
• 5. Define flaw as the soft AND of z_ref and z_self.
• 6. Add a voiced‑gap feature to distinguish stutters from pauses.
• 7. Tune thresholds on JFK, freeze, and test on Reagan and an unseen voice.

Доказательства

Version 1 scored each word by how far it departed from the reference, as a robust z‑score (median/MAD) after removing your overall offset.

Источник

False alarms on clean different‑speaker recordings went from 41/min to 3/min.

Источник

I built Podium, a tool that compares your reading of a speech with a great delivery of the same text (JFK, Reagan) and tells you exactly where you drift and why.

Источник

I also had the same texts read by open Piper TTS voices, with flaws injected into those too. That's the "different speaker" test.

Источник