Kimi K3 Deployment: I’d Measure Accepted-Task Cost Before Renting GPUs

Description and use

This use‑case explains how to evaluate Kimi K3 deployment by measuring accepted‑task cost for self‑hosting versus hosted API, using real workload metrics, pricing, and license review.

Problem

Deciding whether to self-host Kimi K3 or use a hosted API requires accurate cost and performance comparison, accounting for workload, hardware, and licensing constraints.

Outcome

The evaluation framework enables teams to compare self‑hosting versus hosted K3 deployment on a per‑accepted‑task cost basis, incorporating real‑world workload metrics and licensing constraints.

Workflow

• 1. Define representative workload (30‑50 tasks covering coding, long‑context, vision, tool‑calling).
• 2. Deploy K3 on a minimal 8‑GPU cluster using vLLM 0.27.0+ and appropriate RDMA/NVLink settings.
• 3. Run the workload for at least one week, recording input tokens, cache‑hit/miss, output tokens, latency, retries, errors, and acceptance rate.
• 4. Calculate hosted API cost using Moonshot’s pricing tiers (cache‑hit, cache‑miss, output).
• 5. Compute self‑hosted cost per month (cluster hours × hourly rate + engineering, networking, storage).
• 6. Determine break‑even accepted tasks = monthly self‑hosted cost ÷ hosted cost per accepted task.
• 7. Review K3 license for commercial thresholds and attribution requirements.
• 8. Decide on self‑hosting or hosted API based on cost, performance, and licensing fit.

Evidence

K3 is a 2.8‑trillion‑parameter Mixture‑of‑Experts model with 104 billion activated parameters per token

Source

Moonshot’s published prices are $0.30 per 1M cache‑hit input tokens, $3.00 per 1M cache‑miss input tokens, and $15.00 per 1M output tokens

Source

The custom license grants broad rights to use, copy, modify, fine‑tune, deploy, distribute, sublicense, and sell the software and weights

Source