Your auto-captioner is measuring the wrong thing: a ratio that fired on a perfectly good clip

Описание и применение

Автоматическое добавление субтитров к AI‑видео: решение проблемы несинхронности и слишком длинных субтитров через выравнивание текста и тайминга, а также проверку скорости речи.

Задача

Automatic captioning of AI‑generated ad videos produced out‑of‑sync or overly long subtitles due to mismatched timing and text sources.

Результат

Accurate subtitles that match the spoken content and timing, eliminating long static captions and mis‑aligned text.

Рабочий процесс

• Generate video clip
• Extract audio and run Whisper transcription to get timing and text
• Align script lines with Whisper segments using word‑count weighting
• Generate .ass file with aligned timings and script text
• Burn subtitles with ffmpeg
• Apply speech‑rate guard: if any segment’s words‑per‑second < 1.5, fallback to hand‑written cues

Доказательства

Take timing from the audio, take text from the script.

Источник

If any single segment covers more than 40% of the clip, distrust the timing

Источник