How I Built a Unified API Gateway for 200+ AI Models (Architecture Deep Dive)

Description and use

SarangAI consolidates 200+ AI models into a single OpenAI‑compatible endpoint, normalizes responses and streaming, handles billing and rate limits, and offers a CLI for terminal use.

Problem

Managing multiple AI providers requires separate accounts, billing, SDKs, and rate limits, creating a bottleneck for developers experimenting with new models.

Outcome

SarangAI eliminates the need for multiple provider accounts, unifies API interactions, normalizes responses and streaming, and provides real‑time billing and rate limiting, enabling developers to experiment with 200+ models through a single endpoint.

Workflow

• Receive request at API Gateway
• Router selects target model based on request
• Provider Adapter translates request to provider format
• Provider returns response
• Adapter normalizes response to OpenAI format
• Billing & Rate Limiter updates prepaid balance and enforces limits
• Client receives normalized response or SSE stream

Evidence

If you want to use 4 different models, you have to manage 4 accounts, 4 invoices, and 4 sets of documentation.

Source

I built SarangAI to solve this. It's a unified AI gateway that routes all requests to 200+ models through a single OpenAI-compatible endpoint.

Source

The solution: adapter pattern. Each provider has an adapter that: Translates the request from OpenAI format to the provider's format; Translates the response from the provider's format back to OpenAI format; Handles errors and retry logic.

Source

In SarangAI, I normalize all streaming to the same SSE format as OpenAI.

Source

For this, I use a combination of Redis (for fast balance checks) and a database (for audit trails).

Source