I built a "Can I run this LLM?" site where every VRAM number is measured, not guessed

说明与用途

ModelFit 通过测量 GGUF 文件大小并精确计算 KV 缓存需求，为大型语言模型提供准确的 VRAM 要求，消除了估算冲突。

问题

Five popular pages gave conflicting VRAM estimates for Kimi K3, all using the same formula that ignores key components such as embedding tables, higher‑precision attention tensors, output heads, and container overhead.

结果

ModelFit delivers precise VRAM requirements for each quantization of large LLMs, resolving conflicting estimates and enabling users to determine GPU suitability quickly.

工作流

• 1. fetch.py walks the Hugging Face API to collect trending GGUF repos, downloads, and keywords; it sums shard sizes and pulls config.json for each repo.
• 2. build.py computes total memory need as weights + KV cache + overhead, renders a page per model with a quant‑by‑quant table, GPU verdicts, an interactive VRAM checker, and FAQ with JSON‑LD.
• 3. KV cache size is exact when config.json is present (using GQA formula) or estimated with a flat +20%/min‑1.5 GB heuristic, clearly indicated on the page.
• 4. A daily cron re‑fetches data, re‑tests against llama.cpp reference numbers, rebuilds, deploys, and pings IndexNow so new models appear within 24 h.

证据

Five popular pages gave me: 594 GB, 1.51 TB, 1.68 TB, "you can't run it, period", one page said Q4 fits in "about 800 GB", another said 1.2 TB

来源

Every GGUF quant of every popular model is already published as a file on Hugging Face. The file size IS the answer — it's what you download, what loads into memory.

来源

KV cache is exact when config.json is available (GQA formula: 2 × layers × kv_heads × head_dim × ctx × 2 bytes), otherwise a flat +20%/min-1.5GB heuristic — and the page says which one it used.

来源

A daily cron re-fetches, re-tests (formula gate against llama.cpp reference numbers), rebuilds, deploys, and pings IndexNow. New model release → page live within 24h, usually before the blog posts.

来源