Inside goose-swarm: How We Turned One Local Model Into a Self‑Verifying Fleet

Описание и применение

Система goose‑swarm позволяет управлять локальной LLM‑флотом без прямого обращения к модели, обеспечивая детерминированное планирование, быстрый отклик на сбои и самоверификацию задач.

Задача

Running a local LLM fleet efficiently while ensuring each worker stays healthy, avoids deadlocks, and self‑verifies its outputs without external model calls.

Результат

The system achieved deterministic, reproducible scheduling across a local LLM fleet, reduced stall detection time from minutes to seconds, and enabled self‑verification without external model calls.

Рабочий процесс

• 1. Define three core traits (TaskDispatcher, Judge, Replanner) to abstract model interactions.
• 2. Implement the scheduler loop with a single shared State, a fan‑out‑based max‑heap, and drain‑then‑refill claiming.
• 3. Use a work‑stealing device picker that prioritizes least‑loaded nodes and speeds for hard tasks.
• 4. Attach a watchdog that times out on silent streams (default 900s) and classifies stalls as transient or content‑retry.
• 5. Run parallel planning: fast structural draft on a smart model, fleet‑parallel prose detailing, and a pure‑Rust merge for deterministic DAGs.
• 6. Deploy on three qwopus3.6‑27b models via LM Studio and LM Link, enabling self‑verification and healing.
• 7. Validate with unit tests that never call a model, ensuring concurrency rules and reproducibility.
• 8. Monitor event logs for stuck events and auto‑requeue failed tasks with appropriate hints.

Доказательства

The concurrency core is decoupled from any LLM by three traits, so the whole scheduler is unit‑testable against mocks — no model, no network.

Источник

The judge needs no model for the verdicts that can kill: it counts tool calls and file writes, so 'explores forever, writes nothing' is caught in ~150s, not 7 minutes.

Источник

Every gate is engineered to be inconclusive on a missing tool or timeout, so it only ever reds on a genuine, reproducible defect.

Источник