I built an open‑source tool to turn your random home GPUs & Ollama into an OpenAI‑compatible cloud (no port forwarding)

Описание и применение

Viento позволяет использовать локальные GPU‑LLM как удалённые OpenAI‑совместимые эндпоинты без проброса портов, используя исходящий WebSocket‑туннель к облачному шлюзу.

Задача

Developers with home GPUs running Ollama or vLLM struggled to access them remotely from laptops or cloud‑hosted projects due to complex port forwarding, unreliable tunnels, or costly cloud GPU rentals.

Результат

Viento enables developers to expose local GPU‑based LLMs as OpenAI‑compatible endpoints without port forwarding, reducing setup complexity, improving security, and eliminating cloud GPU costs.

Рабочий процесс

• Install Viento via pip: pip install viento
• Run the Viento node daemon on the local machine
• Configure your application to use the Viento endpoint (e.g., https://viento.onrender.com/v1/chat/completions)
• The Viento gateway forwards requests over an outbound WebSocket to the local engine
• Tokens are streamed back via SSE, with automatic backpressure and queueing handled by the gateway

Доказательства

Viento runs as a tiny background node daemon on your machine.

Источник

It opens an outbound WebSocket tunnel (wss://) to a lightweight cloud gateway mesh.

Источник

Whenever your app makes a standard OpenAI-format API call, the gateway routes the job down through the websocket to your local engine.

Источник