How to Meter Node.js Speech-to-Text API Work: 429 Headers and Tenant Queues

Description and use

Implemented a robust, per-tenant queueing system for speech-to-text transcription that correctly handles rate limiting (429) and distinguishes between retryable errors and configuration failures, enabling accurate cost attribution and improved system stability.

Problem

Directly sending sales-call transcription requests to a speech-to-text API mixes customer latency with provider capacity, risks exhausting concurrency budgets with noisy tenants, and obscures per-tenant cost attribution for AI bills.

Outcome

Implemented a robust, per-tenant queueing system for speech-to-text transcription that correctly handles rate limiting (429) and distinguishes between retryable errors and configuration failures, enabling accurate cost attribution and improved system stability.

Workflow

• Receive audio upload and create a job with tenantId, audioUrl, and audioMinutes in a 'pending' state.
• Worker picks up pending jobs and sends them to the speech-to-text API.
• If the API returns 429, parse the Retry-After header (integer seconds or HTTP-date) to determine the minimum delay.
• If Retry-After is absent or invalid, use capped exponential backoff with jitter.
• If the API returns a non-429 4xx error, mark the job as 'failed' and persist the response body for diagnosis, as this indicates a capability or configuration issue.
• Update job state to 'complete' upon success or 'failed' after max retries, allowing the CRM UI to reflect accurate status.
• Use Infrai API discovery to check ASR readiness at startup if applicable.

Evidence

The simplest design sends an uploaded call directly to a speech-to-text API and waits. That design mixes customer latency with provider capacity, gives a noisy tenant the whole concurrency budget, and makes an HTTP request lifecycle responsible for work that may take much longer.

Source

Treat 429 as a scheduling response. If Retry-After is present, parse either its integer-seconds form or its HTTP-date form and make that the minimum delay before the next attempt.

Source