RelayPilot: An AI Night-Shift Engineer for LLM Relay Stations

Description and use

RelayPilot is an AI ops copilot for LLM relay station operators that automatically monitors channels, detects suspicious key usage, answers natural‑language queries, and sends concise Telegram alerts.

Problem

Relay stations that sit between users and model providers struggle to monitor channel health, detect key misuse, answer ops questions, and alert operators without alert fatigue.

Outcome

RelayPilot reduced the time to detect a key leak from hours to minutes, prevented a simulated quota‑burn incident, and provided operators with concise, actionable alerts and natural‑language queries.

Workflow

• Probe each upstream channel on a schedule to collect latency, error rates, and quota burn.
• Run a local Qwen3 model via Ollama to classify per‑key usage windows as normal or suspicious.
• If a key is classified as suspicious, automatically quarantine it and ping the owner via Telegram.
• Answer natural‑language queries using the same local model and channel data.
• Send Telegram alerts for critical events, with quiet hours and a daily digest for non‑critical events.

Evidence

RelayPilot is an AI ops copilot for people who run LLM relay stations — the small, often one-person services that sit between users and model providers, juggling API keys, channel quotas, and bills.

Source

It does four things: Channel health monitoring, Automatic quarantine, Natural-language ops queries, Telegram alerts.

Source

A small open-weight model (Qwen3 via Ollama) classifies per-key usage windows — request rate, token profile, time-of-day, geography — into normal/suspicious.

Source

The model only classifies; all money-adjacent decisions (quarantine thresholds, quota math, billing) are plain code.

Source