Comparing LLMs for Voice Assistants with Oxlo

说明与用途

一份逐步的 Python 脚本，能够录音、通过 Oxlo Whisper 进行转录、基于延迟和字数对多种 LLM 进行基准测试、清理 TTS 友好输出，并为语音助手挑选最佳模型。

问题

Choosing the fastest, most natural LLM for a voice assistant requires a repeatable benchmark that measures latency, word count, and TTS‑friendly output.

结果

The script identifies Llama 3.3 70B as the fastest, most concise model for the sample weather query, demonstrating a reproducible method to compare LLMs for voice assistants on Oxlo.ai.

工作流

• Record 5 s of audio with sounddevice and save as query.wav
• Send query.wav to Oxlo Whisper endpoint and retrieve transcript
• Define a system prompt that forces plain‑text, no‑markdown responses
• For each candidate model: send user message, measure latency, count words, collect raw text
• Clean each raw text by removing asterisks, backticks, and excess whitespace
• Score each result by latency plus word count penalty, sort, and pick the best
• Print transcript, per‑model metrics, and the winning response

证据

I keep the entire pipeline on Oxlo.ai so billing and routing stay simple.

来源

I test Llama 3.3 70B, Qwen 3 32B, Kimi K2.6, and DeepSeek V3.2.

来源