Will it fit on my GPU? I built a calculator that finally answers it

Description and use

StudioTV’s VRAM calculator accurately estimates whether a large language model fits on a given GPU, accounting for attention type, precision, and context length, and offers deployment and cost‑comparison guidance.

Problem

Determining whether a large language model will fit on a specific GPU given its memory constraints, precision, and context length.

Outcome

The calculator accurately predicts memory usage across diverse attention mechanisms, enabling users to choose appropriate precision, context length, and hardware, and to compare on‑premise deployment with cloud rental costs.

Workflow

• 1. Select model, workload, context length, request concurrency, GPU, and engine.
• 2. View calculator output: required GPUs, fit status, memory breakdown, speed, and cost.
• 3. Explore weight‑format vs. context‑length grid to find optimal precision and token limits.
• 4. Review rental prices and obtain ready‑to‑paste launch commands for chosen engine.
• 5. If out of memory, use offload options such as MoE or layer‑level offloading.

Evidence

The usual answer is a back‑of‑the‑envelope formula: parameters times bytes per weight, plus a KV cache computed as if every layer looked at the whole context.

Source

The calculator reads each model's configuration layer by layer and computes the cache the way vLLM and llama.cpp actually allocate it, including FP8 cache formats and the extra window llama.cpp keeps on sliding‑window layers.

Source

In this example, Gemma 4 31B in Q4_K_M takes 17.1 GiB of weights, 3.7 GiB of KV cache for one 32K-token request and about 3 GiB of runtime overhead: 23.7 GiB out of the 28.8 GiB usable on the RTX 5090.

Source