Benchmarking FunASR Streaming ASR: Throughput, Memory Overhead, and p99 Latency Under Load
Description and use
Benchmarking FunASR under 50 concurrent WebSocket clients revealed throughput limits, memory usage, and p99 latency, informing production readiness.
Problem
Determine if FunASR can sustain real‑time streaming under concurrent load before allocating infrastructure.
Outcome
Under 50 concurrent clients, FunASR achieved a sustained throughput of X RPS (exact value omitted due to truncation), with a memory increase from Y MB to Z MB and a p99 latency of W ms. The benchmark isolated inference overhead from network routing by comparing to a B‑Lost LLM relay baseline.
Workflow
- Deploy FunASR in a Docker container on an 8‑core VM with 16 GB RAM
- Configure FunASR to use paraformer‑zh in streaming mode with VAD and punctuation
- Prepare 200 Mandarin call clips (15‑45 s, 16 kHz mono WAV)
- Write an async Python harness using aiohttp to simulate 50 concurrent WebSocket clients
- Stream each client 4 audio files, sending 200 ms chunks and signaling end with is_speaking:false
- Collect TTFT, total time, and chunk count per request
- Measure memory before and after the load test with psutil
- Calculate RPS and p99 latency from collected metrics
Evidence
When evaluating ModelScope's FunASR for production speech recognition pipelines, the marketing materials promise real-time streaming ASR with low latency. But how does it actually perform under concurrent load? I stress-tested the FunASR server with a realistic multi-client harness to measure throughput ceiling, memory behavior, and tail latency before committing infrastructure budget.
FunASR server: Docker container on 8-core VM, 16GB RAM, no GPU acceleration
Model: paraformer-zh streaming mode with VAD and punctuation enabled
Load generator: Python 3.11 async harness with aiohttp, simulating 50 concurrent WebSocket clients
Audio corpus: 200 real-world Mandarin customer service call clips (15-45 sec each, 16kHz mono WAV)
- Record date: 2026-10-19T20:41:35.787Z