Improving Desktop Screen Perception with Neo v0.1 Vision Engine

说明与用途

Neo v0.1通过在多显示器高DPI环境中提供准确的视觉定位和OCR，提升桌面自动化的可靠性，能够精准交互UI元素。

问题

Desktop automation requires accurate visual grounding and OCR text detection, especially in multi-monitor high‑DPI setups.

结果

Neo v0.1 enhances LuxurAI Cowork’s desktop automation by providing reliable visual grounding and OCR in multi‑monitor high‑DPI setups, enabling accurate UI interactions at 15 fps.

工作流

• Capture live desktop frames at 15 fps using dual‑GPU cluster
• Apply Windows UI Automation (UIA) to identify UI elements
• Use low‑level Win32 APIs to interact with applications (type, click)
• Perform OCR on captured frames for text detection
• Integrate results into LuxurAI Cowork automation workflows

证据

Neo v0.1 runs on our dedicated dual‑GPU cluster, processing live desktop frames at 15 fps.

来源

Desktop automation requires accurate visual grounding and OCR text detection, especially in multi-monitor high-DPI setups.

来源