Palimpsest: C++ Tool for Extracting Text from Scrolling Screen Recordings

Description and use

Palimpsest is a C++ tool for extracting text from screen recordings of scrolling documents. It uses 'dwell gating' to select only stable frames, achieving 94.56% accuracy and 323 FPS on a laptop. The tool runs locally, ensuring data privacy.

Problem

Extracting text from screen recordings of scrolling documents (e.g., lectures, banking regulations) is difficult because generic OCR tools fail on video, cloud services are expensive and privacy-invasive, and manual transcription is time-consuming. Existing tools do not handle the specific challenge of deduplicating overlapping frames in scrolling content.

Outcome

Palimpsest successfully extracts text from scrolling screen recordings with 94.56% accuracy on dense banking regulatory text. It achieves 323 FPS on a laptop iGPU by using a lock-free pipeline and 'dwell gating' to process only stable frames, significantly outperforming naive frame-by-frame OCR approaches.

Workflow

• Clone the Palimpsest repository from GitHub.
• Run the compile.sh script to build the tool and its dependencies (OpenCV, Tesseract, etc.) via CMake FetchContent.
• Execute the tool on the target video file (e.g., ./build/bin/palimpsest recording.mkv).
• Optionally configure parameters such as --stable-dwell-ms for frame stability, --crop for region of interest, and --ignore for watermark removal.
• Review the generated .txt or .docx output for minor errors.

Evidence

It runs at 323 FPS on a laptop iGPU and gets 94.56% accuracy on dense banking regulatory text.

Source

From 21,000 frames in a 5 minute 50 second video, only 5 frames were forwarded to OCR in normal operation.

Source