Inside goose-swarm: How We Turned One Local Model Into a Self‑Verifying Fleet

说明与用途

goose‑swarm 通过模型无关的调度核心，实现了本地 LLM 集群的确定性调度、快速故障检测和任务自我验证。

问题

Running a local LLM fleet efficiently while ensuring each worker stays healthy, avoids deadlocks, and self‑verifies its outputs without external model calls.

结果

The system achieved deterministic, reproducible scheduling across a local LLM fleet, reduced stall detection time from minutes to seconds, and enabled self‑verification without external model calls.

工作流

• 1. Define three core traits (TaskDispatcher, Judge, Replanner) to abstract model interactions.
• 2. Implement the scheduler loop with a single shared State, a fan‑out‑based max‑heap, and drain‑then‑refill claiming.
• 3. Use a work‑stealing device picker that prioritizes least‑loaded nodes and speeds for hard tasks.
• 4. Attach a watchdog that times out on silent streams (default 900s) and classifies stalls as transient or content‑retry.
• 5. Run parallel planning: fast structural draft on a smart model, fleet‑parallel prose detailing, and a pure‑Rust merge for deterministic DAGs.
• 6. Deploy on three qwopus3.6‑27b models via LM Studio and LM Link, enabling self‑verification and healing.
• 7. Validate with unit tests that never call a model, ensuring concurrency rules and reproducibility.
• 8. Monitor event logs for stuck events and auto‑requeue failed tasks with appropriate hints.

证据

The concurrency core is decoupled from any LLM by three traits, so the whole scheduler is unit‑testable against mocks — no model, no network.

来源

The judge needs no model for the verdicts that can kill: it counts tool calls and file writes, so 'explores forever, writes nothing' is caught in ~150s, not 7 minutes.

来源

Every gate is engineered to be inconclusive on a missing tool or timeout, so it only ever reds on a genuine, reproducible defect.

来源