I'm not getting up: a voice remote for YouTube that runs fully offline (and understands Pidgin)

Description and use

The system enables voice control of YouTube playback on a Mac, running fully offline and supporting Pidgin commands.

Problem

Controlling YouTube playback from a couch without a remote, keyboard, or internet access.

Outcome

The system achieves 28/30 correct command recognition, with 91% accuracy on unseen Pidgin phrases, and an average latency of ~190 ms when thinking mode is disabled.

Workflow

• 1. User says wake word "Hey Jarvis".
• 2. openWakeWord detects wake word and starts recording.
• 3. webrtcvad detects end of speech and stops recording.
• 4. Whisper (mlx-whisper small) transcribes audio to text.
• 5. Gemma 4 E2B via Ollama parses text and returns JSON command.
• 6. AppleScript sends command to Chrome to control YouTube player.
• 7. A pixel sloth icon appears to confirm command reception.

Evidence

It controls YouTube in Chrome on my Mac. No remote, no keyboard, no getting up. Everything runs on my laptop: no internet, no account, no API key.

Source

I tested 30 phrases on two local models. 22 of them were phrases the model had never seen as examples, including Pidgin I made up on the spot.

Source

Gemma 4's two misses both returned <code>none</code>, so it just didn't act. Gemma 3's misses were actions: it paused the video when I said "tell me a joke", and when I said "make we go back two minutes" it went forward two minutes.

Source