I built a Mandarin to English caption pipeline

Описание и применение

Создана система Finchling для синхронного распознавания мандаринского языка и генерации английских субтитров с сохранением исходного текста.

Задача

Need to speak Mandarin while drafting English text and keep original words visible.

Результат

Finchling achieves a median 95.79 ms latency from audio end to final English commit for three recorded clips.

Рабочий процесс

• Install dependencies via README command
• Run finchling with --live --ui for microphone input
• Use --push-to-talk or recognizer endpoints to commit clauses
• Optionally enable --jev for typed questions
• Use --early-commit to allow speculative completion
• Handle conflicts if later recognition revises committed prefix

Доказательства

I wanted to speak Mandarin while drafting English text and keep the original words visible.

Источник

It reads microphone blocks or a WAV file, shows the recognition hypothesis beside an English preview, and emits completed clauses as commit events.

Источник

Three upstream recorded clips measured a 95.79 ms median from the end of each audio file to its final English commit.

Источник

The README provides one command to install the speech extras and download models. Python 3.10 or newer and uv are required. Speech recognition uses sherpa-onnx. Translation uses Argos and CTranslate2 on the CPU.

Источник