Building a Small Multilingual Offline Voice‑Control Pipeline with ONNX

Описание и применение

Создана компактная офлайн‑система распознавания ключевых слов для устройств IoT, использующая модели ONNX и платформу Voicute. Система обеспечивает низкую задержку, низкое потребление памяти и батареи, а также отсутствие зависимости от облака.

Задача

Deploying a lightweight, always‑listening voice‑control system on resource‑constrained devices without relying on continuous cloud transcription.

Результат

The KWS pipeline achieved low false‑activation rates, high recall, and acceptable latency on ESP32‑S3, enabling battery‑powered, offline voice control for multilingual commands.

Рабочий процесс

• Define fixed command set
• Generate initial KWS model (multi‑keyword if needed)
• Test model with real speakers on target microphone
• Deploy model to device and run on‑device inference
• Collect misses and false activations
• Iteratively add hard negatives or real‑voice samples
• Re‑train or fine‑tune model
• Validate final performance on hardware

Доказательства

For this kind of fixed vocabulary, keyword spotting (KWS) can be a better fit than running a general ASR model all the time.

Источник

The model is only one part of a usable always‑listening system. A raw score should not automatically trigger a product action.

Источник

We measure at least: true‑trigger recall; false activations per hour; distance and speaking volume; different speakers and accents; TV, music, fan and room noise; model size and device memory; end‑to‑end latency and power impact.

Источник

For teams without a speech ML group, a practical validation workflow is: define the exact fixed phrases; generate a first model; test it with real speakers before downloading; run it on the target microphone and device; collect misses and false activations; add real‑voice samples or hard negatives when needed.

Источник