Kimi K3 Deployment: I’d Measure Accepted-Task Cost Before Renting GPUs

说明与用途

本案例说明如何通过测量接受任务成本来评估 Kimi K3 的部署，比较自托管与托管 API，结合真实工作负载指标、定价和许可证审查。

问题

Deciding whether to self-host Kimi K3 or use a hosted API requires accurate cost and performance comparison, accounting for workload, hardware, and licensing constraints.

结果

The evaluation framework enables teams to compare self‑hosting versus hosted K3 deployment on a per‑accepted‑task cost basis, incorporating real‑world workload metrics and licensing constraints.

工作流

• 1. Define representative workload (30‑50 tasks covering coding, long‑context, vision, tool‑calling).
• 2. Deploy K3 on a minimal 8‑GPU cluster using vLLM 0.27.0+ and appropriate RDMA/NVLink settings.
• 3. Run the workload for at least one week, recording input tokens, cache‑hit/miss, output tokens, latency, retries, errors, and acceptance rate.
• 4. Calculate hosted API cost using Moonshot’s pricing tiers (cache‑hit, cache‑miss, output).
• 5. Compute self‑hosted cost per month (cluster hours × hourly rate + engineering, networking, storage).
• 6. Determine break‑even accepted tasks = monthly self‑hosted cost ÷ hosted cost per accepted task.
• 7. Review K3 license for commercial thresholds and attribution requirements.
• 8. Decide on self‑hosting or hosted API based on cost, performance, and licensing fit.

证据

K3 is a 2.8‑trillion‑parameter Mixture‑of‑Experts model with 104 billion activated parameters per token

来源

Moonshot’s published prices are $0.30 per 1M cache‑hit input tokens, $3.00 per 1M cache‑miss input tokens, and $15.00 per 1M output tokens

来源

The custom license grants broad rights to use, copy, modify, fine‑tune, deploy, distribute, sublicense, and sell the software and weights

来源