How I built Browy, an AI agent that lives in your browser

Описание и применение

Browy – это AI‑агент, работающий в вашем браузере, использующий ваш профиль и аутентификацию для выполнения задач на веб‑страницах, включая клики, заполнение форм и выполнение JavaScript, при этом обеспечивая контроль над опасными инструментами.

Задача

Develop an AI agent that can interact with web pages in the user’s existing browser session, leveraging the user’s authentication and data without launching a separate browser instance.

Результат

Browy enables authenticated web automation within the user’s existing browser, eliminating the need for a separate headless browser and preserving login state. It achieves this through a native messaging host, Copilot SDK integration, and a comprehensive set of browser tools, while allowing users to enable or disable potentially risky tools.

Рабочий процесс

• 1. Install Browy extension from Chrome Web Store.
• 2. Run a one‑line PowerShell/terminal command to register the native host.
• 3. Open the side panel or DevTools panel to chat with the agent.
• 4. Agent sends a snapshot of the active tab (title, URL, viewport, scroll, selected text, DOM snapshot).
• 5. Agent calls a tool (e.g., click_index, fill_form) via the Copilot SDK.
• 6. Host forwards DevTools Protocol commands to the browser via chrome.debugger.
• 7. Browser executes actions and returns results to the host.
• 8. Host reports changes (urlChanged, domChanged, modalAppeared, elementStateChanged) back to the agent.
• 9. Agent updates chat history and repeats for subsequent messages.

Доказательства

Browy is an AI agent that lives in your browser, and I mean that literally.

Источник

When you send Browy its first message, Chrome puts up a bar saying that Browy started debugging this browser.

Источник

The host is a Node program, and it's where the agent lives. It wraps GitHub's Copilot SDK, which starts the third program, the Copilot CLI, in headless mode and talks JSON-RPC to it over another pair of pipes.

Источник

Browy has 54 browser tools, from click_index and fill_form to get_network_requests and replay_request. Most have a switch in Browy's settings.

Источник

Browy sidesteps that by staying in your browser. Ask about the pull request in front of you, and it reads the pull request in front of you, in the tab where you're already signed in.

Источник