What broke when I shipped a local Whisper dictation app to real Windows hardware

Описание и применение

Автор создал приложение VoxScribe, использующее локальный Whisper для распознавания речи. Проблемы с драйверами и частотой дискретизации микрофонов, особенно Bluetooth, приводили к сбоям. Решения включают выбор WASAPI‑устройства, запись на нативной частоте с последующим ресэмплингом, переход на hold‑to‑talk и буферизацию первых 0,5 с. Теперь приложение стабильно работает на разных микрофонах.

Задача

Audio hardware and driver issues caused failures in a local Whisper dictation app on Windows.

Результат

The app now reliably transcribes speech from various microphones on Windows, handling sample rate differences, Bluetooth latency, and voice activity detection issues.

Рабочий процесс

• 1. Detect and select the WASAPI default input device instead of the generic default.
• 2. Record audio at the device’s native sample rate.
• 3. Resample recorded audio to 16 kHz using linear interpolation before feeding it to Whisper.
• 4. Replace automatic VAD with a hold‑to‑talk mechanism; keep VAD code for future use.
• 5. Optionally keep the mic stream open and buffer the last half‑second to avoid clipping the first word.

Доказательства

The first was a Bluetooth headset that recorded near silence. On my machine the sounddevice library picked a default input device that landed on an MME driver, and for my headset that produced audio so quiet that Whisper transcribed almost nothing.

Источник

The fix was to ask for the WASAPI default input device explicitly instead of trusting the generic default.

Источник

The second was a crash when forcing 16 kHz. Whisper wants 16 kHz audio, so my first version opened the microphone at 16 kHz. That worked on my headset, then failed with an invalid sample rate error on a laptop's internal mic running at 48 kHz.

Источник

The fix was to record at the device's own native rate and resample to 16 kHz in software afterward.

Источник

I switched to hold-to-talk, so the user decides when speech starts and stops and there is nothing to guess.

Источник

I added an opt-in mode that keeps the stream open and holds the last half second in a small in-memory buffer, so a recording starts with audio that is already there.

Источник