Voice AI Audio Compatibility: A 12-Case Phone/Web Acceptance Test

Description and use

A 12‑case acceptance suite verifies that audio traversing phone, browser, and AI voice model matches the declared media contract, preventing hidden conversions and codec errors.

Problem

Ensuring that audio data exchanged between phone, browser, and AI voice model conforms to the exact media contract to avoid subtle audio defects.

Outcome

The 12‑case acceptance suite detects contract mistakes before voice model evaluation, preventing audio defects such as wrong sample rates, codec mismatches, or hidden conversions.

Workflow

• 1. Define a contract row for each boundary (e.g., Phone provider → adapter, Adapter → speech model, etc.).
• 2. Record the expected metadata (codec, sample rate, channels, framing, envelope) for each boundary.
• 3. Capture the actual payload at each boundary using packet capture or stream inspection.
• 4. Validate that the captured payload matches the declared metadata (e.g., Base64‑decoded μ‑law samples at 8 kHz).
• 5. Verify that any conversion (e.g., μ‑law to linear PCM, resampling) is explicit and documented.
• 6. Ensure the outbound payload to the phone playback matches the provider’s required envelope and raw audio bytes.
• 7. Record and label any recordings or observability sinks with container, codec, rate, channels, and timestamp origin.
• 8. Release the integration only when all boundaries meet the contract and no implicit conversions are detected.

Evidence

A sample rate is not a codec, a codec is not a container, and metadata is not a conversion.

Source

For a Twilio bidirectional Media Stream, the contract is explicit: the start.mediaFormat reports audio/x-mulaw, 8,000 Hz, one channel; outbound audio is raw μ-law at 8,000 Hz, Base64 inside a text JSON WebSocket message, with no audio-file header.

Source

The public information baseline is already strong on individual formats. Twilio documents its message envelope and raw outbound μ-law payload. MDN describes WebRTC codec options. SignalWire documents its own stream choices, including PCMU, A-law and linear PCM options, and distinguishes phone streams from browser WebRTC.

Source