Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Fine-tuning Pocket TTS for Castilian Spanish: stop at step 600 and judge the accent by ear

说明与用途

Pocket TTS 在西班牙加泰罗尼亚语上进行微调,在第600步停止并使用EMA权重,保持了清晰度和正确口音,同时避免了词语丢失和错误。

问题

The base Spanish Pocket TTS model sounded like Latin America, not Castilian Spain, and training longer degraded quality.

结果

The fine‑tuned model at step 600 with EMA weights delivers Castilian Spanish narration on CPU, matching base intelligibility and avoiding word errors, while keeping generation speed acceptable.

工作流

  • 1. Choose 24‑layer Spanish model for speed trade‑off.
  • 2. Fine‑tune on ~2,000 Castilian recordings using AMD GPU ROCm.
  • 3. Monitor validation loss; stop at step 600 where loss 0.4951.
  • 4. Release EMA weights instead of raw checkpoint.
  • 5. Deploy with pocket-tts serve --config hf://inferencekey/pocket-tts-spanish-24l-castilian/config.yaml.
  • 6. Generate audio per complete sentence, spell numbers out.
  • 7. Evaluate with Qwen3‑ASR to catch dropped/invented words.
  • 8. Have a human listen to each clip to confirm accent.
  • 9. Iterate if ASR fails or accent unsatisfactory.

证据

The base Spanish model was perfectly intelligible. The accent gave it away: foreign‑sounding, with seseo, where the Castilian "z" and soft "c" collapse into an "s" as in much of Latin America.

来源

Validation loss is the error the model makes on held‑out clips it never trains on; lower is better. Ours hit its minimum at step 600 (0.4951) and had drifted up to 0.5095 by step 1500.

来源

From step 600 we released the EMA weights instead of the raw ones. EMA (exponential moving average) blends the weights over recent steps, so it moves further from the base model more gently.

来源

First, send complete sentences or short paragraphs. The model ends each chunk with a falling, final intonation, so a fragmented paragraph sounds stop‑and‑go.

来源

Second, spell numbers out. The model handles "siete y cuarto" every time; "7:15" sometimes trips it.

来源

  • 记录日期: 2026-10-20T21:56:09.483Z