Comparing LLMs for Voice Assistants with Oxlo

Description and use

A step‑by‑step Python script that records audio, transcribes via Oxlo Whisper, benchmarks multiple LLMs for latency and brevity, cleans TTS‑friendly output, and selects the best model for a voice assistant.

Problem

Choosing the fastest, most natural LLM for a voice assistant requires a repeatable benchmark that measures latency, word count, and TTS‑friendly output.

Outcome

The script identifies Llama 3.3 70B as the fastest, most concise model for the sample weather query, demonstrating a reproducible method to compare LLMs for voice assistants on Oxlo.ai.

Workflow

• Record 5 s of audio with sounddevice and save as query.wav
• Send query.wav to Oxlo Whisper endpoint and retrieve transcript
• Define a system prompt that forces plain‑text, no‑markdown responses
• For each candidate model: send user message, measure latency, count words, collect raw text
• Clean each raw text by removing asterisks, backticks, and excess whitespace
• Score each result by latency plus word count penalty, sort, and pick the best
• Print transcript, per‑model metrics, and the winning response

Evidence

I keep the entire pipeline on Oxlo.ai so billing and routing stay simple.

Source

I test Llama 3.3 70B, Qwen 3 32B, Kimi K2.6, and DeepSeek V3.2.

Source