Building a Small Multilingual Offline Voice‑Control Pipeline with ONNX

Description and use

A lightweight offline keyword‑spotting pipeline using ONNX and Voicute was built for IoT devices, delivering low latency, minimal memory and battery usage, and no cloud dependency.

Problem

Deploying a lightweight, always‑listening voice‑control system on resource‑constrained devices without relying on continuous cloud transcription.

Outcome

The KWS pipeline achieved low false‑activation rates, high recall, and acceptable latency on ESP32‑S3, enabling battery‑powered, offline voice control for multilingual commands.

Workflow

• Define fixed command set
• Generate initial KWS model (multi‑keyword if needed)
• Test model with real speakers on target microphone
• Deploy model to device and run on‑device inference
• Collect misses and false activations
• Iteratively add hard negatives or real‑voice samples
• Re‑train or fine‑tune model
• Validate final performance on hardware

Evidence

For this kind of fixed vocabulary, keyword spotting (KWS) can be a better fit than running a general ASR model all the time.

Source

The model is only one part of a usable always‑listening system. A raw score should not automatically trigger a product action.

Source

We measure at least: true‑trigger recall; false activations per hour; distance and speaking volume; different speakers and accents; TV, music, fan and room noise; model size and device memory; end‑to‑end latency and power impact.

Source

For teams without a speech ML group, a practical validation workflow is: define the exact fixed phrases; generate a first model; test it with real speakers before downloading; run it on the target microphone and device; collect misses and false activations; add real‑voice samples or hard negatives when needed.

Source