How I built Live Voice: one tool contract, six realtime voice providers

说明与用途

Live Voice 将六个实时语音提供商统一到一个工具合同下，提供一致的用户界面和安全的 Cloudflare Worker 中继。

问题

Integrating multiple real‑time voice AI providers into a single mobile app while keeping a consistent tool‑based workflow and UI behavior.

结果

Live Voice now supports six real‑time providers with a single tool contract, consistent UI behavior, and a secure hosted relay, enabling users to log nutrition and exercise via voice across iOS and Android.

工作流

• Define a single JSON contract listing 25 tool actions with declared effects (read, navigate, write, destructive).
• Implement adapters for each provider that translate between provider‑specific message formats and the contract.
• Ensure both iOS and Android apps bundle the same contract and CI verifies consistency.
• Add UI logic that interprets tool effects: show Undo for writes, require confirmation for destructive actions, and enforce user confirmation for writes when the setting is on.
• Deploy a Cloudflare Worker that acts as a WebSocket relay for the Plus/Pro plan, performing user‑id checks, rate limiting, and billing by time.
• Provide a Metal/AGSL orb UI element that visualizes listening/thinking/speaking states based on audio loudness.
• Publish the app and source code, inviting community feedback and PRs.

证据

The assistant can only act through tools. They live in one file, shared/voice‑assistant/tools.json: 25 actions like open_camera, capture_photo, scan_barcode, log_food, log_water, start_fast, log_workout, update_setting and end_conversation.

来源

Both apps bundle an identical copy, and a CI script fails the build if either copy drifts.

来源

The hosted relay… checks the user id, rate limits, the RevenueCat plan and the remaining voice balance.

来源

The glowing orb is a Metal shader on iOS and an AGSL port on Android 13+, with a Canvas fallback on older versions.

来源