Palimpsest: C++ Tool for Extracting Text from Scrolling Screen Recordings

Описание и применение

Palimpsest — это инструмент на C++ для извлечения текста из видеозаписей экрана с прокруткой документов. Он использует механизм 'dwell gating' для выбора только стабильных кадров, что обеспечивает точность 94,56% и скорость 323 FPS на ноутбуке. Инструмент работает локально, защищая конфиденциальность данных.

Задача

Extracting text from screen recordings of scrolling documents (e.g., lectures, banking regulations) is difficult because generic OCR tools fail on video, cloud services are expensive and privacy-invasive, and manual transcription is time-consuming. Existing tools do not handle the specific challenge of deduplicating overlapping frames in scrolling content.

Результат

Palimpsest successfully extracts text from scrolling screen recordings with 94.56% accuracy on dense banking regulatory text. It achieves 323 FPS on a laptop iGPU by using a lock-free pipeline and 'dwell gating' to process only stable frames, significantly outperforming naive frame-by-frame OCR approaches.

Рабочий процесс

• Clone the Palimpsest repository from GitHub.
• Run the compile.sh script to build the tool and its dependencies (OpenCV, Tesseract, etc.) via CMake FetchContent.
• Execute the tool on the target video file (e.g., ./build/bin/palimpsest recording.mkv).
• Optionally configure parameters such as --stable-dwell-ms for frame stability, --crop for region of interest, and --ignore for watermark removal.
• Review the generated .txt or .docx output for minor errors.

Доказательства

It runs at 323 FPS on a laptop iGPU and gets 94.56% accuracy on dense banking regulatory text.

Источник

From 21,000 frames in a 5 minute 50 second video, only 5 frames were forwarded to OCR in normal operation.

Источник