ome-projects/ome — Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle m…

Description and use

Open Model Engine (OME) is an open‑source Kubernetes operator that simplifies the deployment, scaling, and lifecycle management of large language models (LLMs). It provides built‑in GPU scheduling and integrates with popular inference runtimes such as SGLang, vLLM, TensorRT‑LLM, and Triton, enabling seamless, production‑grade serving of LLMs on Kubernetes clusters.

Problem

ome-projects/ome helps add AI capability without starting from a blank model stack.

Outcome

Medium: useful for prototypes or internal workflows, with dependency and maintenance checks before production.

Workflow

• prototype AI features
• build RAG or agent workflows
• evaluate model tooling

Evidence

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

Source