Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

I built a hands-free slide remote for a friend who can never find the right page

说明与用途

演讲者在大型 PDF 演示文稿中经常失去幻灯片位置。使用 MediaPipe 手势跟踪、Whisper 语音识别和 Gemma 进行语义幻灯片选择,构建了本地离线遥控器,提升了隐私和成本效益。

问题

Presenters lose track of slide position in large PDFs during talks, making navigation stressful and error-prone.

结果

The remote reliably navigates slides using gestures and voice, with a hybrid architecture that reduces reliance on LLMs for simple commands and improves safety by preferring no‑action over incorrect jumps.

工作流

  • Extract slide text from PDF/PPTX
  • Index slides
  • Track hand via MediaPipe from webcam
  • Detect gestures (swipe, open palm, fist) and map to slide actions
  • Record audio via microphone
  • Transcribe with Whisper using slide titles as hints
  • Detect command phrases
  • Apply deterministic logic for simple commands
  • Apply keyword + fuzzy matching for content requests
  • If multiple candidates, send to Gemma via Ollama for final decision
  • Execute slide navigation or no‑action if confidence low

证据

The webcam tracks the presenter's hand using MediaPipe.

来源

Whisper runs locally, so the microphone audio does not need to be uploaded to a speech API.

来源

Gemma via Ollama → semantic understanding and slide selection

来源

  • 记录日期: 2026-10-04T15:40:47.096Z