Mobile UI Control for AI Agents

说明与用途

AI 代理现在可以通过本地运行的 OpenClaw 网关查看并控制手机 UI。

问题

Enabling AI agents to interact with mobile device interfaces by providing visual perception and touch control.

结果

AI agents can now directly interact with mobile apps, enabling automated UI testing and task execution on real devices.

工作流

• 1. Install the OpenClaw Android gateway on the target phone.
• 2. Run the gateway locally to expose a control interface.
• 3. Connect the AI agent to the gateway endpoint.
• 4. Agent captures screen snapshots and sends tap commands to the phone.
• 5. Phone executes the gestures and updates the UI accordingly.

证据

this project gives AI Agents UI control, it lets it see, tap and interact on your phone

来源

also i ran the gateway locally on my phone

来源