The accessibility tree gave 162 refs before any screenshot

说明与用途

agent-desktop 在 macOS 上通过可访问性树获取了 162 个 Finder 交互元素，无需截图，展示了更安全、可拒绝的自动化方法。

问题

Desktop automation agents rely on pixel-based coordinate math, which can fail silently after UI changes.

结果

The tool successfully retrieved 162 interactive elements from Finder via the accessibility tree without screenshots, demonstrating a safer, rejection‑aware approach compared to pixel math.

工作流

• npm install -g agent-desktop
• agent-desktop version
• agent-desktop permissions --guide
• agent-desktop list-apps
• agent-desktop snapshot --app Finder -i
• agent-desktop screenshot --app Finder <path>

证据

agent-desktop snapshot --app Finder -i returned ok=true, error=null, ref_count=162 interactive elements, complete=false, window title Finder

来源

The permission census from agent-desktop permissions --guide read accessibility GRANTED (already on for my terminal automation chain), automation UNKNOWN, screen_recording DENIED

来源

agent-desktop list-apps returned zero apps headless, consistent with their documented prohibition on foreground activation for headless calls

来源