Build a Real‑Time Voice AI Agent with LiveKit, Pulse STT, and Lightning TTS

说明与用途

使用 LiveKit、Pulse STT 和 Lightning TTS 构建低延迟语音代理，STT、LLM 和 TTS 并行工作，降低响应时间。

问题

Real‑time voice agents suffer from high latency when each stage (STT, LLM, TTS) waits for the previous to finish, making the user experience sluggish.

结果

Implemented a low‑latency voice agent where STT, LLM, and TTS operate concurrently, reducing end‑to‑end response time and improving user experience.

工作流

• 1. Set up LiveKit (cloud or self‑hosted) and obtain credentials.
• 2. Create a Smallest AI account and generate an API key.
• 3. Install Python 3.11, create a virtual environment, and install livekit‑plugins‑smallestai, livekit‑plugins‑openai, livekit‑plugins‑silero, python‑dotenv.
• 4. Store SMALLEST_API_KEY, LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET, and OPENAI_API_KEY in environment variables.
• 5. Write a VoiceAgent class extending livekit.agents.Agent, configuring instructions, prewarm VAD, and entrypoint to load credentials.
• 6. In entrypoint, create an AgentSession with VAD, STT (smallestai.STT), LLM (openai.LLM), and TTS (smallestai.TTS).
• 7. Run the agent via livekit CLI or programmatically, ensuring LiveKit handles media plumbing and streaming.
• 8. Test latency by measuring time from user speech to audible response, adjusting models or parameters as needed.

证据

LiveKit handles the real‑time media and agent plumbing. Your STT and TTS choices still determine how quickly speech becomes usable text and how quickly generated text becomes audible speech.

来源

The pipeline is concurrent, not sequential

来源