Unified LLM API for Multi-Provider Chatbot Fallback: Automatic Failover for CRM Actions

Описание и применение

Унифицированный API LLM обеспечивает надёжное преобразование транскриптов звонков в действия CRM, автоматически переключаясь между поставщиками и гарантируя согласованность схемы ответа.

Задача

An edtech in-app chatbot must reliably convert sales‑call transcripts into CRM actions while handling provider outages and rate limits.

Результат

The unified API approach reduces token costs by sending compact summaries, ensures consistent CRM action contracts, and provides transparent failover without frontend changes.

Рабочий процесс

• 1. Send transcript to unified LLM API with a strict prompt for a single JSON action.
• 2. Retry up to 4 times on transient 429 errors, backing off exponentially.
• 3. Parse and validate the JSON against required keys (lead_status, course, next_action, owner, follow_up_at).
• 4. If validation fails, raise an error and log the request ID for debugging.
• 5. On success, forward the action to the CRM with an idempotency key to prevent duplicates.
• 6. Log provider failure and retry state for observability.

Доказательства

Keep the frontend speaking one request shape, validate the returned action object, and retry only when the failure is transient.

Источник

The pass criteria should be mechanical: The response parses as JSON and contains every required key.

Источник

A standard queue is at-least-once delivery; treating it as exactly once is how duplicate follow-up tasks reach admissions staff.

Источник