Deploying Portkey - Open-Source AI Gateway for LLM Routing

Description and use

Deploying Portkey, an open-source AI gateway, using Docker Compose and Traefik for TLS termination and Basic Auth.

Problem

Integrating with multiple LLM providers requires separate SDKs and complex routing logic.

Outcome

A secure, stateless AI gateway is running on the server, exposing a single OpenAI‑compatible API endpoint and a protected web console.

Workflow

• Create project directory and navigate to it
• Obtain server public IP with curl -4 ifconfig.me
• Create .env with DOMAIN, LETSENCRYPT_EMAIL, SERVER_IP
• Install htpasswd utility via sudo apt install -y apache2-utils
• Generate .htpasswd credentials with htpasswd -c ~/portkey/.htpasswd portkeyadmin
• Create docker-compose.yml with Traefik and Portkey services
• Configure Traefik labels for HTTPS, Basic Auth, IP allowlist, and routing rules
• Run docker compose up -d to start services
• Verify containers are running

Evidence

Portkey is an open‑source AI gateway that unifies access to 250+ Large Language Models (LLMs) across dozens of providers behind a single, OpenAI‑compatible API.

Source

This guide deploys Portkey on a Linux server using Docker Compose and secures it with Traefik as a reverse proxy for TLS termination, protecting access with Basic Authentication for the web console and an IP allowlist for the API endpoint.

Source