Invisible characters in 2,392 Android apps' translations: Weblate's zero-width-space fix shows up in what ships

Description and use

Invisible formatting characters in Android translations cause bugs. Weblate reduces zero‑width spaces by a factor of four but not other controls. A new l10n‑guard warning now flags these issues in CI.

Problem

Invisible formatting characters such as zero‑width spaces, soft hyphens, byte‑order marks and bidirectional controls are introduced into Android app translations through copy‑and‑paste from translation tools and web pages, breaking search, string comparison and line wrapping.

Outcome

Weblate repositories ship about a quarter as many stray zero‑width spaces per million units compared to non‑Weblate repos (17 vs 70), but they are not cleaner for other control characters (531 vs 303). The study also revealed that many left‑to‑right marks in English pseudo‑locales were due to an unaccounted en‑XC locale.

Workflow

• 1. Identify all F‑Droid repositories and their translation catalogs.
• 2. Classify repositories by whether they use Weblate (493) or not (1,899).
• 3. Count invisible characters that are present in translations but absent in the English source, separating zero‑width spaces (U+200B) and other control‑class characters.
• 4. Compute the ratio of extra characters per million translated units for Weblate and non‑Weblate repos.
• 5. Analyze the distribution of soft hyphens, byte‑order marks, and other stray characters.
• 6. Publish findings and create a new l10n‑guard warning to flag these issues in CI.

Evidence

Translations pick up invisible characters: zero-width spaces, soft hyphens, byte-order marks and bidi controls.

Source

Weblate already deals with one of them. Its RemoveZeroSpace autofix deletes a U+200B that the source string doesn't have (Khmer is exempt).

Source

Weblate repositories ship about a quarter as many stray zero-width spaces, but they are not cleaner on the characters Weblate leaves alone.

Source

These findings shaped a new invisible-char warning in l10n-guard 0.15.0, MV2's open-source (Apache-2.0) translation checker and GitHub Action.

Source