NexusOS — Your Personal Jarvis, Powered by AI

说明与用途

NexusOS 是一个本地 AI 助手，使用 Whisper、PyAutoGUI 和 Playwright，通过语音或文本控制计算机、浏览器、文件和智能家居设备。

问题

Users want a personal assistant that can control their computer, browse the web, manage files, and interact with smart home devices via voice or text without manual input.

结果

NexusOS enables hands-free computer interaction, automating routine tasks, and integrating smart home control, all while preserving privacy by running locally.

工作流

• 1. Say wake word "Hey Nexus" to activate the system.
• 2. Voice or text input is transcribed by Whisper and processed by the backend.
• 3. Depending on the command, the system uses PyAutoGUI for computer control, Playwright for browser tasks, or file system APIs for file management.
• 4. For multi-step routines, the system executes predefined sequences automatically.
• 5. Contextual memory is retrieved from SQLite vector embeddings to inform responses.
• 6. Smart home actions are sent via MQTT to Home Assistant or compatible devices.
• 7. Responses are spoken back using TTS or displayed in the UI.

证据

Voice Pipeline — Listens for a wake word, transcribes with Whisper, and speaks back using TTS. The whole loop takes about 2-3 seconds

来源

Computer Control — Moves your mouse, clicks buttons, and types text using PyAutoGUI

来源

Browser Automation — Built on Playwright, it can open browsers, navigate, fill forms, and read page content back to you

来源