Invisible characters in 2,392 Android apps' translations: Weblate's zero-width-space fix shows up in what ships

说明与用途

Android 翻译中出现的不可见字符导致搜索和字符串比较错误。Weblate 将零宽度空格减少四倍，但对其他控制字符无效。新 l10n‑guard 警告已在 CI 中检测这些问题。

问题

Invisible formatting characters such as zero‑width spaces, soft hyphens, byte‑order marks and bidirectional controls are introduced into Android app translations through copy‑and‑paste from translation tools and web pages, breaking search, string comparison and line wrapping.

结果

Weblate repositories ship about a quarter as many stray zero‑width spaces per million units compared to non‑Weblate repos (17 vs 70), but they are not cleaner for other control characters (531 vs 303). The study also revealed that many left‑to‑right marks in English pseudo‑locales were due to an unaccounted en‑XC locale.

工作流

• 1. Identify all F‑Droid repositories and their translation catalogs.
• 2. Classify repositories by whether they use Weblate (493) or not (1,899).
• 3. Count invisible characters that are present in translations but absent in the English source, separating zero‑width spaces (U+200B) and other control‑class characters.
• 4. Compute the ratio of extra characters per million translated units for Weblate and non‑Weblate repos.
• 5. Analyze the distribution of soft hyphens, byte‑order marks, and other stray characters.
• 6. Publish findings and create a new l10n‑guard warning to flag these issues in CI.

证据

Translations pick up invisible characters: zero-width spaces, soft hyphens, byte-order marks and bidi controls.

来源

Weblate already deals with one of them. Its RemoveZeroSpace autofix deletes a U+200B that the source string doesn't have (Khmer is exempt).

来源

Weblate repositories ship about a quarter as many stray zero-width spaces, but they are not cleaner on the characters Weblate leaves alone.

来源

These findings shaped a new invisible-char warning in l10n-guard 0.15.0, MV2's open-source (Apache-2.0) translation checker and GitHub Action.

来源