From Burned-In Text to Bilingual SRT: How Subtitle OCR, Local Whisper and AI Translation Fit Together

Description and use

Step‑by‑step workflow for extracting hardcoded subtitles via OCR, transcribing audio locally with Whisper, translating contextually, and producing bilingual SRT files.

Problem

Extract usable subtitles from videos that either have hardcoded subtitles or no subtitles at all, and produce bilingual subtitles.

Outcome

The workflow produces accurate, bilingual subtitle files with preserved timing, reducing manual correction effort and enabling local processing without external billing.

Workflow

• 1. Sample frames from the video, e.g., 4 fps, cropped to subtitle area using ffmpeg.
• 2. Preprocess frames (grayscale, contrast boost, thresholding) to improve OCR accuracy.
• 3. Run OCR to detect and recognize subtitle text per frame.
• 4. Deduplicate OCR results across consecutive frames, merge runs, and build accurate start/end timestamps.
• 5. If no subtitles, extract audio, convert to 16kHz mono, and run Whisper locally to generate an SRT file.
• 6. Re-segment Whisper output into readable subtitle cues, adjusting for sentence boundaries and character limits.
• 7. Translate each cue in context windows, ensuring the same cue count and order, respecting reading speed, and using a glossary for names.
• 8. Combine original and translated cues into a bilingual subtitle format.
• 9. Polish timings and text in a bilingual editor, then export the final SRT.

Evidence

Hardcoded subtitles are just pixels. To turn them back into an .srt file you need to answer two questions for every line: what does it say and when is it on screen.

Source

A minimal run with the open-source CLI looks like this: ffmpeg -i input.mp4 -ar 16000 -ac 1 audio.wav whisper audio.wav --model medium --language ja --output_format srt

Source

Translate in context windows. Send a block of consecutive cues together (with their indexes) so the model can see the full sentence and the surrounding conversation, then ask for the same number of cues back.

Source

These days I run the same pipeline in GeekLink, a desktop app for Mac and Windows that does each of these steps in one place.

Source