FitGuru: Building a Private, On-Device AI Workout Coach in Kotlin

说明与用途

FitGuru 展示了手机如何在不将视频上传到云端的情况下进行运动姿势分析和重复计数，使用本地 MediaPipe Pose 和 Gemini 进行后期分析。

问题

How to coach workout form and count reps on a phone without sending video to the cloud?

结果

FitGuru demonstrates that a phone can coach squats and other exercises, count reps accurately, provide real‑time corrections, and give a post‑set debrief—all while keeping video on the device.

工作流

• Open app and select lift
• Place phone to capture user
• CameraX streams frames to MediaPipe Pose
• Reflex Engine computes joint angles and runs state machines for rep counting and form cues
• Spoken corrections are delivered via TTS or ElevenLabs voice during the set
• User says "finish set" to trigger Cognitive Engine
• Cognitive Engine (Gemini 2.5 Flash) processes summary JSON and generates a spoken debrief
• History is stored in Room database
• Optional ElevenLabs audio is used if API key provided

证据

We built a phone-side workout coach at Innovhacks 4.0 - MediaPipe pose, joint-angle state machines, Gemini debriefs, and ElevenLabs voice - without sending your camera feed to the cloud.

来源

If the camera feed leaves the phone, we already lost.

来源

A squat is not a guess. A rep exists only when the knee and hip cross a “down” threshold and then return past an “up” threshold.

来源