smg-project/smg — Engine-agnostic LLM gateway in Rust.

Description and use

smg-project/smg is an engine‑agnostic LLM gateway written in Rust that provides full OpenAI and Anthropic API compatibility across a wide range of back‑ends (vLLM, TRT‑LLM, TokenSpeed, SGLang, Gemini, etc.) while adding advanced features such as gRPC pipelines, KV‑cache‑aware routing, chat‑history handling, tokenization caching, embeddings, WASM plugins, MCP support, and multi‑tenant auth. This enables teams to turn isolated prompts and tool calls into repeatable, multi‑agent workflows with standardized memory and tool‑use pipelines, simplifying evaluation via clear API/SDK/CLI signals. With strong recent activity, a growing ecosystem (429★, 129 forks, updated 2026‑08‑01), and high production‑readiness signals, it is suitable for a serious pilot after a final review of license, security, and maintainer health.

Problem

smg-project/smg helps turn isolated prompts and tools into repeatable agent workflows.

Outcome

High for an OSS candidate: recent activity, adoption, and ecosystem signals are strong enough for a serious pilot.

Workflow

• coordinate multi-agent workflows
• add tool-use pipelines
• standardize agent memory

Evidence

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

Source