Whisper typed its own prompt into my document

Описание и применение

Добавление предварительного фильтра Silero VAD устраняет ложные транскрипции начального запроса в Whisper, сохраняя реальную речь.

Задача

Whisper can transcribe an initial_prompt as spoken text when audio contains only loud noise, producing phantom sentences in dictation tools.

Результат

Adding a Silero VAD pre‑filter eliminates phantom prompt transcriptions while preserving real speech, with a 0.3 threshold yielding no noise passes and minimal word loss.

Рабочий процесс

• Record a short audio burst (e.g., fan noise).
• Run Silero VAD on the burst before Whisper decoding.
• If VAD returns no speech, discard the burst and log discard_reason="no_speech".
• Otherwise, pass the burst to Whisper for transcription.

Доказательства

If you use faster-whisper or openai-whisper with an initial_prompt, try this sometime. Record two seconds of a fan, or a keyboard, or nothing in particular, loud enough to pass whatever loudness check you have. Transcribe it.

Источник

The fix is to stop asking Whisper whether there was speech, and ask an independent detector before decoding.

Источник

Silero's default speech threshold is 0.5. I used 0.3, and here is the evidence, including its limits: At 0.3: 0 words dropped, 0 noise bursts passed.

Источник