What I learned building speech tech for a language with zero datasets

Description and use

Building TTS and ASR for Crimean Tatar without existing data: shift to forced alignment, select a base model containing the key phoneme, and fine‑tune on a small dataset.

Problem

Building TTS and ASR for Crimean Tatar without any existing speech data.

Outcome

By reframing the problem as forced alignment and selecting a base model that already contains the critical phoneme, the author built a Crimean Tatar TTS and ASR system with only ~15 h of data, achieving significant error reductions and correct pronunciation of /q/ without needing large corpora.

Workflow

• 1. Identify that the problem is alignment, not transcription, because audiobooks provide ground‑truth text.
• 2. Run forced alignment on large audio blocks; detect silent failures by listening.
• 3. Split audio into smaller units (chapters) to expose failures early.
• 4. Locate chapters in the book using character‑level matching to handle agglutinative morphology and dual scripts.
• 5. Choose a base TTS model that already contains the target phoneme (/q/) by inspecting multilingual training data (Kazakh, Uyghur, Bashkir).
• 6. Map Crimean Tatar orthography to the base model’s known letters (e.g., къ→ҡ).
• 7. Fine‑tune the model with the small aligned dataset (≈15 h).
• 8. Evaluate ASR performance (WER/CER) before and after fine‑tuning and decoding tweaks.
• 9. Debug alignment timestamp drift by comparing old vs. new timestamps with ASR.
• 10. Iterate until alignment quality and pronunciation meet expectations.
• 11. Deploy the model for downstream applications.

Evidence

"I stopped trying to transcribe and started trying to align: known text, audio, find where each word lands, cut on those boundaries."

Source

"The first big alignment run covered seven and a half hours of audio… I got a bit under two of usable material…"

Source

"The base model had already decided… In the base model, before it ever saw a word of my language, "q" was already wired to /k/ by English, Spanish and French."

Source

"The multilingual base I ended up choosing had roughly 1500 hours of Kazakh, 400 of Uyghur and 250 of Bashkir…"

Source

"The corpus kept growing: 5.9 hours, then 11.5, then 15.3… The automatic metric flatlined…"

Source