huggingface_hub v2.3.0 week: how I fixed a Windows crash in hf cache verify

Описание и применение

На Windows команда hf cache verify падала, когда вывод перенаправлялся в файл из‑за недопустимого символа ❌ в кодировке cp1252. Заменив его на [FAIL] через _ascii_safe, команда теперь корректно выводит список неверных файлов и завершает работу с кодом 1.

Задача

On Windows, the hf cache verify command crashed when its output contained the ❌ emoji and was redirected to a pipe or file, because the terminal encoding (UTF‑8) differed from the pipe encoding (cp1252) which cannot represent the emoji.

Результат

The patch replaces the ❌ emoji with [FAIL] when stdout cannot encode it, restoring correct output to pipes and CI logs on Windows while keeping the same exit behavior.

Рабочий процесс

• 1. Reproduce the crash by running hf cache verify with output piped on Windows.
• 2. Identify that the crash is due to cp1252 encoding lacking the ❌ emoji.
• 3. Locate the _ascii_safe helper in cli/_output.py and understand its fallback mechanism.
• 4. Add a call to _ascii_safe("❌", "[FAIL]") in cli/cache.py to replace the emoji with a safe fallback.
• 5. Run regression tests with CliRunner(charset="cp1252") to confirm the bug is fixed.
• 6. Submit a PR (#5255) with the change and regression test.
• 7. Review and merge the PR on October 27, 2026.
• 8. Verify that the command now outputs [FAIL] and the list of bad files when piped.
• 9. Optionally set PYTHONUTF8=1 to force UTF‑8 stdout if using an older release.

Доказательства

On Windows, Python writes to a terminal in UTF-8 but to a pipe, a file or a CI log in the old ANSI code page, cp1252.

Источник

The fix prints [FAIL] when the stream cannot encode the emoji.

Источник

After #5255 the command asks _ascii_safe whether stdout can encode ❌ before printing.

Источник

The same pipe now prints the report, and the exit code is still 1:

Источник