Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Fine-tuning Pocket TTS for Castilian Spanish: stop at step 600 and judge the accent by ear

Описание и применение

Финетюнинг модели Pocket TTS на кастильском испанском завершился на шаге 600 с использованием EMA‑весов, сохранив ясность речи и правильный акцент, при этом избегая потери слов и генерации ошибок.

Задача

The base Spanish Pocket TTS model sounded like Latin America, not Castilian Spain, and training longer degraded quality.

Результат

The fine‑tuned model at step 600 with EMA weights delivers Castilian Spanish narration on CPU, matching base intelligibility and avoiding word errors, while keeping generation speed acceptable.

Рабочий процесс

  • 1. Choose 24‑layer Spanish model for speed trade‑off.
  • 2. Fine‑tune on ~2,000 Castilian recordings using AMD GPU ROCm.
  • 3. Monitor validation loss; stop at step 600 where loss 0.4951.
  • 4. Release EMA weights instead of raw checkpoint.
  • 5. Deploy with pocket-tts serve --config hf://inferencekey/pocket-tts-spanish-24l-castilian/config.yaml.
  • 6. Generate audio per complete sentence, spell numbers out.
  • 7. Evaluate with Qwen3‑ASR to catch dropped/invented words.
  • 8. Have a human listen to each clip to confirm accent.
  • 9. Iterate if ASR fails or accent unsatisfactory.

Доказательства

The base Spanish model was perfectly intelligible. The accent gave it away: foreign‑sounding, with seseo, where the Castilian "z" and soft "c" collapse into an "s" as in much of Latin America.

Источник

Validation loss is the error the model makes on held‑out clips it never trains on; lower is better. Ours hit its minimum at step 600 (0.4951) and had drifted up to 0.5095 by step 1500.

Источник

From step 600 we released the EMA weights instead of the raw ones. EMA (exponential moving average) blends the weights over recent steps, so it moves further from the base model more gently.

Источник

First, send complete sentences or short paragraphs. The model ends each chunk with a falling, final intonation, so a fragmented paragraph sounds stop‑and‑go.

Источник

Second, spell numbers out. The model handles "siete y cuarto" every time; "7:15" sometimes trips it.

Источник

  • Дата записи: 2026-10-20T21:56:09.483Z