Self-Host a Qwen3.8-27B-Uncensored LLM on Oracle Cloud with Your Own Domain and integrate with AI coding agents

Описание и применение

Развертывание модели Qwen 27B на Oracle Cloud с открытым API совместимым с OpenAI, веб‑чатом и автоматическим HTTPS, преодолевая проблемы брандмауэра и DNS.

Задача

Deploy a 27B parameter Qwen model on an Oracle Cloud VM, expose it via an OpenAI-compatible API, and provide a web chat interface with automatic HTTPS, while overcoming firewall, DNS, and rate‑limit challenges.

Результат

Successfully ran a 27B Qwen model on Oracle Cloud, exposed it via an OpenAI‑compatible API, and provided a web chat interface with automatic HTTPS, overcoming firewall and DNS hurdles.

Рабочий процесс

• Log in via SSH and update system
• Determine VM specs and public IP
• Install llama.cpp and add to PATH
• Start tmux session and run llama serve with flags: --host 127.0.0.1 --port 8080 --api-key <key> --parallel 1 -c 65536 -t 24
• Detach tmux, test locally with curl for health and chat completion
• Create Cloudflare DNS A record for subdomain pointing to VM IP with DNS‑only proxy
• Verify DNS resolution with dig and ifconfig.me
• Open ports 80 and 443 in OCI security list (TCP, 0.0.0.0/0)
• Configure Ubuntu iptables to accept TCP 80/443 above REJECT rule or delete REJECT
• Install and configure Caddy to proxy 127.0.0.1:8080 and obtain Let’s Encrypt cert
• Deploy web chat interface on same domain

Доказательства

Oracle Cloud gave me the 1 month trial server, llama.cpp gave me the model server, and Caddy handled HTTPS.

Источник

When it says listening on ..., stop it with Ctrl+C and start the version you'll keep: llama serve -hf JonathanColetti/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M --host 127.0.0.1 --port 8080 --api-key iamkingjanak --parallel 1 -c 65536 -t 24

Источник

With the orange proxy on, browsers see Cloudflare's addresses and you get Error 522: Connection timed out, and Let's Encrypt validates against Cloudflare instead of your server.

Источник