LLM continuous batching host scheduler in Rust (191k tok/s on 1 vCPU)

说明与用途

LLM 连续批处理主机调度器使用 Rust 编写，单个 vCPU 可实现 191k 令牌/秒，消除工作流中的手动操作。先在原型中测试其兼容性和更新，然后将其集成到可重复的操作任务中。该项目适用于内部原型和工作流，但在正式使用前请核实许可证、维护和文档。

实际用途

LLM continuous batching host scheduler in Rust (191k tok/s on 1 vCPU) helps remove repetitive manual operations from a workflow.

• remove manual work
• connect tools into repeatable flows
• schedule operational tasks

限制与审查说明

Medium: useful for prototypes or internal workflows, with dependency and maintenance checks before production.