Deploying Portkey - Open-Source AI Gateway for LLM Routing

Описание и применение

Развертывание Portkey, открытого шлюза для LLM, с помощью Docker Compose и Traefik, обеспечивающего TLS и базовую аутентификацию.

Задача

Integrating with multiple LLM providers requires separate SDKs and complex routing logic.

Результат

A secure, stateless AI gateway is running on the server, exposing a single OpenAI‑compatible API endpoint and a protected web console.

Рабочий процесс

• Create project directory and navigate to it
• Obtain server public IP with curl -4 ifconfig.me
• Create .env with DOMAIN, LETSENCRYPT_EMAIL, SERVER_IP
• Install htpasswd utility via sudo apt install -y apache2-utils
• Generate .htpasswd credentials with htpasswd -c ~/portkey/.htpasswd portkeyadmin
• Create docker-compose.yml with Traefik and Portkey services
• Configure Traefik labels for HTTPS, Basic Auth, IP allowlist, and routing rules
• Run docker compose up -d to start services
• Verify containers are running

Доказательства

Portkey is an open‑source AI gateway that unifies access to 250+ Large Language Models (LLMs) across dozens of providers behind a single, OpenAI‑compatible API.

Источник

This guide deploys Portkey on a Linux server using Docker Compose and secures it with Traefik as a reverse proxy for TLS termination, protecting access with Basic Authentication for the web console and an IP allowlist for the API endpoint.

Источник