The HWP‑to‑Markdown Problem Nobody Talks About (and a FreeType Gotcha)

Описание и применение

Проблема конвертации файлов .hwp в Markdown, распространённая в Корее, решена инструментом doc2md, который использует rhwp-python и корректно обрабатывает таблицы, а также устраняет сбои из‑за двойной загрузки FreeType.

Задача

Korean .hwp files are common in government and enterprise, but there is no easy way to convert them to text for LLM pipelines, and font‑rendering libraries can crash services.

Результат

The conversion tool normalizes HWP documents into Markdown, enabling consistent processing across formats and eliminating format‑specific glue code. It also resolves a subtle FreeType loading bug that caused intermittent crashes.

Рабочий процесс

• 1. Use rhwp-python to parse the HWP ZIP container and extract structured text and paragraphs.
• 2. Convert tables by tracking row/column positions to preserve data order.
• 3. Preload the FreeType shared library via ctypes before importing any library that uses it, ensuring a single instance is used.
• 4. Output the extracted content as Markdown.
• 5. Feed the Markdown into the existing RAG/LLM pipeline.
• 6. Publish the tool as a free utility at https://www.knowverse.net/en/util/.

Доказательства

Modern .hwp (and .hwpx) files are structured containers, closer to OOXML than to a flat binary blob.

Источник

The parsing step pulled in a FreeType dependency for font metrics, and it segfaulted intermittently under load — but only on the server, never locally.

Источник