The accessibility tree gave 162 refs before any screenshot

Описание и применение

Инструмент agent-desktop смог получить 162 интерактивных элемента из Finder, используя дерево доступности macOS, без скриншотов, демонстрируя более надёжный подход, чем координатный поиск.

Задача

Desktop automation agents rely on pixel-based coordinate math, which can fail silently after UI changes.

Результат

The tool successfully retrieved 162 interactive elements from Finder via the accessibility tree without screenshots, demonstrating a safer, rejection‑aware approach compared to pixel math.

Рабочий процесс

• npm install -g agent-desktop
• agent-desktop version
• agent-desktop permissions --guide
• agent-desktop list-apps
• agent-desktop snapshot --app Finder -i
• agent-desktop screenshot --app Finder <path>

Доказательства

agent-desktop snapshot --app Finder -i returned ok=true, error=null, ref_count=162 interactive elements, complete=false, window title Finder

Источник

The permission census from agent-desktop permissions --guide read accessibility GRANTED (already on for my terminal automation chain), automation UNKNOWN, screen_recording DENIED

Источник

agent-desktop list-apps returned zero apps headless, consistent with their documented prohibition on foreground activation for headless calls

Источник