Whisper typed its own prompt into my document

Description and use

Adding a Silero VAD pre‑filter removes phantom prompt transcriptions in Whisper while preserving real speech.

Problem

Whisper can transcribe an initial_prompt as spoken text when audio contains only loud noise, producing phantom sentences in dictation tools.

Outcome

Adding a Silero VAD pre‑filter eliminates phantom prompt transcriptions while preserving real speech, with a 0.3 threshold yielding no noise passes and minimal word loss.

Workflow

• Record a short audio burst (e.g., fan noise).
• Run Silero VAD on the burst before Whisper decoding.
• If VAD returns no speech, discard the burst and log discard_reason="no_speech".
• Otherwise, pass the burst to Whisper for transcription.

Evidence

If you use faster-whisper or openai-whisper with an initial_prompt, try this sometime. Record two seconds of a fan, or a keyboard, or nothing in particular, loud enough to pass whatever loudness check you have. Transcribe it.

Source

The fix is to stop asking Whisper whether there was speech, and ask an independent detector before decoding.

Source

Silero's default speech threshold is 0.5. I used 0.3, and here is the evidence, including its limits: At 0.3: 0 words dropped, 0 noise bursts passed.

Source