RelayPilot: An AI Night-Shift Engineer for LLM Relay Stations

说明与用途

RelayPilot 是为 LLM 中继站运营商设计的 AI 运维助手，自动监控通道、检测可疑密钥使用、回答自然语言查询，并通过 Telegram 发送简洁警报。

问题

Relay stations that sit between users and model providers struggle to monitor channel health, detect key misuse, answer ops questions, and alert operators without alert fatigue.

结果

RelayPilot reduced the time to detect a key leak from hours to minutes, prevented a simulated quota‑burn incident, and provided operators with concise, actionable alerts and natural‑language queries.

工作流

• Probe each upstream channel on a schedule to collect latency, error rates, and quota burn.
• Run a local Qwen3 model via Ollama to classify per‑key usage windows as normal or suspicious.
• If a key is classified as suspicious, automatically quarantine it and ping the owner via Telegram.
• Answer natural‑language queries using the same local model and channel data.
• Send Telegram alerts for critical events, with quiet hours and a daily digest for non‑critical events.

证据

RelayPilot is an AI ops copilot for people who run LLM relay stations — the small, often one-person services that sit between users and model providers, juggling API keys, channel quotas, and bills.

来源

It does four things: Channel health monitoring, Automatic quarantine, Natural-language ops queries, Telegram alerts.

来源

A small open-weight model (Qwen3 via Ollama) classifies per-key usage windows — request rate, token profile, time-of-day, geography — into normal/suspicious.

来源

The model only classifies; all money-adjacent decisions (quarantine thresholds, quota math, billing) are plain code.

来源