Benchmarking FunASR Streaming ASR: Throughput, Memory Overhead, and p99 Latency Under Load

Описание и применение

Проверка производительности FunASR при 50 одновременных WebSocket‑клиентах выявила пределы пропускной способности, использование памяти и п99‑задержку, что позволяет оценить готовность к продакшн‑нагрузке.

Задача

Determine if FunASR can sustain real‑time streaming under concurrent load before allocating infrastructure.

Результат

Under 50 concurrent clients, FunASR achieved a sustained throughput of X RPS (exact value omitted due to truncation), with a memory increase from Y MB to Z MB and a p99 latency of W ms. The benchmark isolated inference overhead from network routing by comparing to a B‑Lost LLM relay baseline.

Рабочий процесс

• Deploy FunASR in a Docker container on an 8‑core VM with 16 GB RAM
• Configure FunASR to use paraformer‑zh in streaming mode with VAD and punctuation
• Prepare 200 Mandarin call clips (15‑45 s, 16 kHz mono WAV)
• Write an async Python harness using aiohttp to simulate 50 concurrent WebSocket clients
• Stream each client 4 audio files, sending 200 ms chunks and signaling end with is_speaking:false
• Collect TTFT, total time, and chunk count per request
• Measure memory before and after the load test with psutil
• Calculate RPS and p99 latency from collected metrics

Доказательства

When evaluating ModelScope's FunASR for production speech recognition pipelines, the marketing materials promise real-time streaming ASR with low latency. But how does it actually perform under concurrent load? I stress-tested the FunASR server with a realistic multi-client harness to measure throughput ceiling, memory behavior, and tail latency before committing infrastructure budget.

Источник

FunASR server: Docker container on 8-core VM, 16GB RAM, no GPU acceleration

Источник

Model: paraformer-zh streaming mode with VAD and punctuation enabled

Источник

Load generator: Python 3.11 async harness with aiohttp, simulating 50 concurrent WebSocket clients

Источник

Audio corpus: 200 real-world Mandarin customer service call clips (15-45 sec each, 16kHz mono WAV)

Источник