Why your YouTube transcript scraper returns the wrong language (and how to fix it)

Описание и применение

Скриптеры YouTube часто возвращают субтитры неверного языка, что приводит к ошибкам в ИИ‑пайплайнах. Решение: определить оригинальный язык с помощью авто‑генерируемого трека распознавания речи и выбирать лучшие субтитры на этом языке, при необходимости используя Apify Actor для продакшн‑развертывания.

Задача

Scrapers often return captions in the wrong language, causing silent bugs in AI pipelines that consume transcripts.

Результат

By selecting the auto-generated speech recognition track to determine the original language and then choosing human-made captions in that language, the scraper now reliably returns correct-language transcripts, avoiding silent bugs in downstream AI tasks.

Рабочий процесс

• Identify if video is dubbed and read original audio track flag.
• Use language of auto-generated (speech recognition) track as reliable signal for spoken language.
• Select best captions in that language: prefer human-made, fall back to auto-generated if none.
• Handle cloud IP blocking by using residential proxies and retries.
• Optionally use Apify Actor for production deployment.

Доказательства

9 of those 37 came back in the wrong language

Источник

The problem: YouTube's caption list is not ordered by the original language.

Источник

If you're using the open-source youtube-transcript-api library, step 2 looks roughly like this:

Источник