From Burned-In Text to Bilingual SRT: How Subtitle OCR, Local Whisper and AI Translation Fit Together

说明与用途

从硬编码字幕提取、使用 Whisper 本地转录音频、上下文翻译到双语 SRT 的完整工作流程。

问题

Extract usable subtitles from videos that either have hardcoded subtitles or no subtitles at all, and produce bilingual subtitles.

结果

The workflow produces accurate, bilingual subtitle files with preserved timing, reducing manual correction effort and enabling local processing without external billing.

工作流

• 1. Sample frames from the video, e.g., 4 fps, cropped to subtitle area using ffmpeg.
• 2. Preprocess frames (grayscale, contrast boost, thresholding) to improve OCR accuracy.
• 3. Run OCR to detect and recognize subtitle text per frame.
• 4. Deduplicate OCR results across consecutive frames, merge runs, and build accurate start/end timestamps.
• 5. If no subtitles, extract audio, convert to 16kHz mono, and run Whisper locally to generate an SRT file.
• 6. Re-segment Whisper output into readable subtitle cues, adjusting for sentence boundaries and character limits.
• 7. Translate each cue in context windows, ensuring the same cue count and order, respecting reading speed, and using a glossary for names.
• 8. Combine original and translated cues into a bilingual subtitle format.
• 9. Polish timings and text in a bilingual editor, then export the final SRT.

证据

Hardcoded subtitles are just pixels. To turn them back into an .srt file you need to answer two questions for every line: what does it say and when is it on screen.

来源

A minimal run with the open-source CLI looks like this: ffmpeg -i input.mp4 -ar 16000 -ac 1 audio.wav whisper audio.wav --model medium --language ja --output_format srt

来源

Translate in context windows. Send a block of consecutive cues together (with their indexes) so the model can see the full sentence and the surrounding conversation, then ask for the same number of cues back.

来源

These days I run the same pipeline in GeekLink, a desktop app for Mac and Windows that does each of these steps in one place.

来源