agent-sh/computer-use-linux — Linux desktop control over MCP — AT-SPI, GNOME Shell, Wayland portals, ydotool

Description and use

**Brief Summary (2‑3 sentences)**
*agent‑sh/computer‑use‑linux* is an open‑source Rust library that lets AI assistants control a Linux desktop through the Model Context Protocol (MCP), leveraging AT‑SPI, GNOME Shell, Wayland portals, and ydotool. It provides a standard, low‑latency API/SDK/CLI for exposing real‑world tools and data to LLMs, making it easy to ship MCP servers and unify integrations across Linux environments.

**Value**

• **Bridges the gap** between large language models and native Linux applications, turning the desktop into a programmable interface for AI agents.
• **Standardizes communication** via MCP, so the same integration code can be reused across different assistants, tools, and distributions.
• **Accelerates development** by offering ready‑made bindings for common accessibility and input stacks (AT‑SPI, GNOME Shell, Wayland portals, ydotool), eliminating the need to write custom glue code.

**Practical Adoption Path**

1. **Prototype** – Clone the repo, run the provided CLI or embed the Rust SDK in a small service that registers as an MCP server.
2. **Integrate** – Connect your LLM‑backed assistant (e.g., OpenAI, Anthropic) to the MCP endpoint; use the SDK to send commands like “click button X” or “read window Y”.
3. **Extend** – Add custom handlers for domain‑specific tools (e.g., IDEs, browsers) by leveraging the existing AT‑SPI/Wayland abstractions.
4. **Deploy** – Package the service as a container or systemd unit on target workstations; the standard MCP contract ensures other agents can interoperate without code changes.

**Production Readiness**

• **Active maintenance**: recent commits (as of 2026‑06‑29), 226 stars, 21 forks, and a healthy Rust ecosystem indicate ongoing development.
• **Mature API surface**: clear CLI/SDK, well‑documented signals, and language‑agnostic metadata make integration straightforward.
• **Risk profile**: No major licensing or security red flags identified, though a final review of the license and maintainer responsiveness is advisable.

Overall, the project is mature enough for a serious pilot in production environments, especially where Linux desktop automation by AI agents is required.

Problem

agent-sh/computer-use-linux helps connect AI assistants to real tools and data through a standard protocol.

Outcome

High for an OSS candidate: recent activity, adoption, and ecosystem signals are strong enough for a serious pilot.

Workflow

• connect AI agents to tools
• ship Model Context Protocol servers
• standardize integrations

Evidence

Linux desktop control over MCP — AT-SPI, GNOME Shell, Wayland portals, ydotool

Source