Unified LLM API for Multi-Provider Chatbot Fallback: Automatic Failover for CRM Actions

说明与用途

统一的LLM API 能够可靠地将销售通话记录转换为CRM操作，并在供应商之间自动切换，同时强制执行严格的JSON合同。

问题

An edtech in-app chatbot must reliably convert sales‑call transcripts into CRM actions while handling provider outages and rate limits.

结果

The unified API approach reduces token costs by sending compact summaries, ensures consistent CRM action contracts, and provides transparent failover without frontend changes.

工作流

• 1. Send transcript to unified LLM API with a strict prompt for a single JSON action.
• 2. Retry up to 4 times on transient 429 errors, backing off exponentially.
• 3. Parse and validate the JSON against required keys (lead_status, course, next_action, owner, follow_up_at).
• 4. If validation fails, raise an error and log the request ID for debugging.
• 5. On success, forward the action to the CRM with an idempotency key to prevent duplicates.
• 6. Log provider failure and retry state for observability.

证据

Keep the frontend speaking one request shape, validate the returned action object, and retry only when the failure is transient.

来源

The pass criteria should be mechanical: The response parses as JSON and contains every required key.

来源

A standard queue is at-least-once delivery; treating it as exactly once is how duplicate follow-up tasks reach admissions staff.

来源