Kimi K3 Deployment: I’d Measure Accepted-Task Cost Before Renting GPUs

Описание и применение

Кейс описывает, как оценить стоимость и производительность развертывания модели Kimi K3, сравнивая собственный хостинг и облачный сервис, учитывая реальные метрики нагрузки, цены и лицензионные ограничения.

Задача

Deciding whether to self-host Kimi K3 or use a hosted API requires accurate cost and performance comparison, accounting for workload, hardware, and licensing constraints.

Результат

The evaluation framework enables teams to compare self‑hosting versus hosted K3 deployment on a per‑accepted‑task cost basis, incorporating real‑world workload metrics and licensing constraints.

Рабочий процесс

• 1. Define representative workload (30‑50 tasks covering coding, long‑context, vision, tool‑calling).
• 2. Deploy K3 on a minimal 8‑GPU cluster using vLLM 0.27.0+ and appropriate RDMA/NVLink settings.
• 3. Run the workload for at least one week, recording input tokens, cache‑hit/miss, output tokens, latency, retries, errors, and acceptance rate.
• 4. Calculate hosted API cost using Moonshot’s pricing tiers (cache‑hit, cache‑miss, output).
• 5. Compute self‑hosted cost per month (cluster hours × hourly rate + engineering, networking, storage).
• 6. Determine break‑even accepted tasks = monthly self‑hosted cost ÷ hosted cost per accepted task.
• 7. Review K3 license for commercial thresholds and attribution requirements.
• 8. Decide on self‑hosting or hosted API based on cost, performance, and licensing fit.

Доказательства

K3 is a 2.8‑trillion‑parameter Mixture‑of‑Experts model with 104 billion activated parameters per token

Источник

Moonshot’s published prices are $0.30 per 1M cache‑hit input tokens, $3.00 per 1M cache‑miss input tokens, and $15.00 per 1M output tokens

Источник

The custom license grants broad rights to use, copy, modify, fine‑tune, deploy, distribute, sublicense, and sell the software and weights

Источник