We tested every small model our camera agent uses on a 4-core CPU. The bigger model lost.

Описание и применение

После тестирования моделей на 4‑ядерном CPU без GPU команда OpenNVR выбрала более быстрые и компактные модели, улучшив скорость отклика и надёжность камеры.

Задача

Choosing the right AI models for a camera agent that runs on a 4‑core CPU without a GPU.

Результат

After benchmarking, the team switched to smaller, faster models that perform better on CPU, improving reliability and response time for camera agents.

Рабочий процесс

• Run a single test per model on a 4‑core CPU with no GPU
• Measure pass rate and tool usage for language models
• Count hallucinations and people count accuracy for vision models
• Calculate word error rate and latency for speech‑to‑text models
• Measure real‑time factor for text‑to‑speech models
• Compute mAP and latency for search models
• Measure frame‑rate and accuracy for detection models
• Compare results and update defaults accordingly

Доказательства

We gave it 20 investigation cases, for example "person at the loading bay at 01:14".

qwen3:4b scored less than half of qwen3:1.7b.

Qwen3-VL never said yes to something that wasn’t there.

Parakeet TDT 0.6B v3 (onnx-asr, int8) 1.4% WER 0.31 s.

Kokoro-82M sounds nicer than Piper. But on 4 cores it runs at 1.9x real time, so it takes longer to make the audio than to play it, and replies would pause mid-sentence. Piper runs at 0.23x.

SigLIP 2 B/16 scored 0.672 mAP against CLIP ViT-B/32's 0.571.