Measuring what local AI gets wrong in translated dialogue

Описание и применение

Тестовый набор инструментов на Python проверяет, насколько переведённый диалог подходит по времени и смыслу, выявляя ошибки в синхронизации и терминологии.

Задача

Local open-weight models produce translated dialogue that may not fit timing constraints or preserve meaning, making automated dubbing unreliable.

Результат

The toolkit identified that 4 of 12 translations fit timing, 2 needed stretching, 2 were too short, and 4 were too long. Shorter variants from Gemma improved fit for 3 lines, but timing alone did not guarantee meaning preservation.

Рабочий процесс

• Translate dialogue with Opus‑MT (English→French) or generate shorter variants with Gemma 3 4B via Ollama
• Run localisation‑measurements verifier to check dialogue fit, timing, overlap, drift, terminology
• Review failed lines and adjust or reject them
• Optionally regenerate shorter variants with Gemma
• Repeat verification until all checks pass

Доказательства

Localisation measurements is a Python toolkit with four command-line checks: Estimated dialogue fit inside a time window. Subtitle timing, overlap and reading speed. Latency and accumulated drift budgets. Terminology consistency and figures.

Источник

The core tools use only the Python standard library. A separate example uses Opus-MT for English-to-French translation and Gemma 3 4B through local Ollama for shorter variants.

Источник

The verifier passes 78 cases checking exit codes and hand-calculated results. The example input can deliberately produce a failing quality-check exit code: that means a measurement found a problem.

Источник

Line 008 changes from past to present tense while passing its timing check. All candidates require human review.

Источник