I'm not getting up: a voice remote for YouTube that runs fully offline (and understands Pidgin)

Описание и применение

Система позволяет управлять воспроизведением YouTube с помощью голоса, полностью работая офлайн на Mac, включая команды на пиджин.

Задача

Controlling YouTube playback from a couch without a remote, keyboard, or internet access.

Результат

The system achieves 28/30 correct command recognition, with 91% accuracy on unseen Pidgin phrases, and an average latency of ~190 ms when thinking mode is disabled.

Рабочий процесс

• 1. User says wake word "Hey Jarvis".
• 2. openWakeWord detects wake word and starts recording.
• 3. webrtcvad detects end of speech and stops recording.
• 4. Whisper (mlx-whisper small) transcribes audio to text.
• 5. Gemma 4 E2B via Ollama parses text and returns JSON command.
• 6. AppleScript sends command to Chrome to control YouTube player.
• 7. A pixel sloth icon appears to confirm command reception.

Доказательства

It controls YouTube in Chrome on my Mac. No remote, no keyboard, no getting up. Everything runs on my laptop: no internet, no account, no API key.

Источник

I tested 30 phrases on two local models. 22 of them were phrases the model had never seen as examples, including Pidgin I made up on the spot.

Источник

Gemma 4's two misses both returned <code>none</code>, so it just didn't act. Gemma 3's misses were actions: it paused the video when I said "tell me a joke", and when I said "make we go back two minutes" it went forward two minutes.

Источник