I built a "Can I run this LLM?" site where every VRAM number is measured, not guessed

Описание и применение

ModelFit предоставляет точные данные о VRAM для больших языковых моделей, используя измеренные размеры GGUF‑файлов и точные расчёты KV‑кэша, устраняя разногласия в оценках.

Задача

Five popular pages gave conflicting VRAM estimates for Kimi K3, all using the same formula that ignores key components such as embedding tables, higher‑precision attention tensors, output heads, and container overhead.

Результат

ModelFit delivers precise VRAM requirements for each quantization of large LLMs, resolving conflicting estimates and enabling users to determine GPU suitability quickly.

Рабочий процесс

• 1. fetch.py walks the Hugging Face API to collect trending GGUF repos, downloads, and keywords; it sums shard sizes and pulls config.json for each repo.
• 2. build.py computes total memory need as weights + KV cache + overhead, renders a page per model with a quant‑by‑quant table, GPU verdicts, an interactive VRAM checker, and FAQ with JSON‑LD.
• 3. KV cache size is exact when config.json is present (using GQA formula) or estimated with a flat +20%/min‑1.5 GB heuristic, clearly indicated on the page.
• 4. A daily cron re‑fetches data, re‑tests against llama.cpp reference numbers, rebuilds, deploys, and pings IndexNow so new models appear within 24 h.

Доказательства

Five popular pages gave me: 594 GB, 1.51 TB, 1.68 TB, "you can't run it, period", one page said Q4 fits in "about 800 GB", another said 1.2 TB

Источник

Every GGUF quant of every popular model is already published as a file on Hugging Face. The file size IS the answer — it's what you download, what loads into memory.

Источник

KV cache is exact when config.json is available (GQA formula: 2 × layers × kv_heads × head_dim × ctx × 2 bytes), otherwise a flat +20%/min-1.5GB heuristic — and the page says which one it used.

Источник

A daily cron re-fetches, re-tests (formula gate against llama.cpp reference numbers), rebuilds, deploys, and pings IndexNow. New model release → page live within 24h, usually before the blog posts.

Источник