Self-Host a Qwen3.8-27B-Uncensored LLM on Oracle Cloud with Your Own Domain and integrate with AI coding agents

说明与用途

在 Oracle Cloud 上部署 27B Qwen 模型，公开 OpenAI 兼容 API，提供 Web 聊天界面，并实现自动 HTTPS，同时解决防火墙和 DNS 问题。

问题

Deploy a 27B parameter Qwen model on an Oracle Cloud VM, expose it via an OpenAI-compatible API, and provide a web chat interface with automatic HTTPS, while overcoming firewall, DNS, and rate‑limit challenges.

结果

Successfully ran a 27B Qwen model on Oracle Cloud, exposed it via an OpenAI‑compatible API, and provided a web chat interface with automatic HTTPS, overcoming firewall and DNS hurdles.

工作流

• Log in via SSH and update system
• Determine VM specs and public IP
• Install llama.cpp and add to PATH
• Start tmux session and run llama serve with flags: --host 127.0.0.1 --port 8080 --api-key <key> --parallel 1 -c 65536 -t 24
• Detach tmux, test locally with curl for health and chat completion
• Create Cloudflare DNS A record for subdomain pointing to VM IP with DNS‑only proxy
• Verify DNS resolution with dig and ifconfig.me
• Open ports 80 and 443 in OCI security list (TCP, 0.0.0.0/0)
• Configure Ubuntu iptables to accept TCP 80/443 above REJECT rule or delete REJECT
• Install and configure Caddy to proxy 127.0.0.1:8080 and obtain Let’s Encrypt cert
• Deploy web chat interface on same domain

证据

Oracle Cloud gave me the 1 month trial server, llama.cpp gave me the model server, and Caddy handled HTTPS.

来源

When it says listening on ..., stop it with Ctrl+C and start the version you'll keep: llama serve -hf JonathanColetti/Qwen3.8-27B-Uncensored-GGUF:Q4_K_M --host 127.0.0.1 --port 8080 --api-key iamkingjanak --parallel 1 -c 65536 -t 24

来源

With the orange proxy on, browsers see Cloudflare's addresses and you get Error 522: Connection timed out, and Let's Encrypt validates against Cloudflare instead of your server.

来源