I built an open‑source tool to turn your random home GPUs & Ollama into an OpenAI‑compatible cloud (no port forwarding)

说明与用途

Viento 通过出站 WebSocket 隧道，将本地 GPU LLM 转变为远程 OpenAI 兼容端点，省去端口转发和云 GPU 成本。

问题

Developers with home GPUs running Ollama or vLLM struggled to access them remotely from laptops or cloud‑hosted projects due to complex port forwarding, unreliable tunnels, or costly cloud GPU rentals.

结果

Viento enables developers to expose local GPU‑based LLMs as OpenAI‑compatible endpoints without port forwarding, reducing setup complexity, improving security, and eliminating cloud GPU costs.

工作流

• Install Viento via pip: pip install viento
• Run the Viento node daemon on the local machine
• Configure your application to use the Viento endpoint (e.g., https://viento.onrender.com/v1/chat/completions)
• The Viento gateway forwards requests over an outbound WebSocket to the local engine
• Tokens are streamed back via SSE, with automatic backpressure and queueing handled by the gateway

证据

Viento runs as a tiny background node daemon on your machine.

来源

It opens an outbound WebSocket tunnel (wss://) to a lightweight cloud gateway mesh.

来源

Whenever your app makes a standard OpenAI-format API call, the gateway routes the job down through the websocket to your local engine.

来源