RelayPilot: An AI Night-Shift Engineer for LLM Relay Stations

Описание и применение

RelayPilot — это AI‑операционный копилот для операторов LLM‑релейных станций, автоматически мониторит каналы, выявляет подозрительное использование ключей, отвечает на вопросы на естественном языке и отправляет сжатые уведомления в Telegram.

Задача

Relay stations that sit between users and model providers struggle to monitor channel health, detect key misuse, answer ops questions, and alert operators without alert fatigue.

Результат

RelayPilot reduced the time to detect a key leak from hours to minutes, prevented a simulated quota‑burn incident, and provided operators with concise, actionable alerts and natural‑language queries.

Рабочий процесс

• Probe each upstream channel on a schedule to collect latency, error rates, and quota burn.
• Run a local Qwen3 model via Ollama to classify per‑key usage windows as normal or suspicious.
• If a key is classified as suspicious, automatically quarantine it and ping the owner via Telegram.
• Answer natural‑language queries using the same local model and channel data.
• Send Telegram alerts for critical events, with quiet hours and a daily digest for non‑critical events.

Доказательства

RelayPilot is an AI ops copilot for people who run LLM relay stations — the small, often one-person services that sit between users and model providers, juggling API keys, channel quotas, and bills.

Источник

It does four things: Channel health monitoring, Automatic quarantine, Natural-language ops queries, Telegram alerts.

Источник

A small open-weight model (Qwen3 via Ollama) classifies per-key usage windows — request rate, token profile, time-of-day, geography — into normal/suspicious.

Источник

The model only classifies; all money-adjacent decisions (quarantine thresholds, quota math, billing) are plain code.

Источник