A tool that tells you which local LLM your PC can actually run

Description and use

Jaull is a tool that determines which local language models can run on a specific PC, considering CPU, RAM, GPU, and available backends. It offers up to five concrete plans with exact files, runtimes, and GPU‑offload settings, and provides measured speed metrics.

Problem

Determine which local language models a PC can run given its hardware constraints.

Outcome

Jaull accurately predicts which LLMs fit on a given PC, provides benchmarked speed metrics, and highlights GPU offload inefficiencies.

Workflow

• Detect CPU, RAM, GPU and available backends
• Ask user questions about usage, priorities, languages, commercial intent
• Search Hugging Face metadata for compatible models
• Estimate memory by component (weights, KV cache, overhead, margins) treating RAM and VRAM as separate pools
• Generate up to five concrete plans with file, runtime, backend, GPU offload
• Provide explanations of plan positions
• Allow user to benchmark or score any plan on their machine
• Save results with SHA-256, machine, and protocol for future reordering

Evidence

Jaull is an open-source Python tool with a terminal UI. It detects your CPU, RAM, GPU and backends, asks a few plain questions (what you'll use it for, what matters most, languages, commercial use) and searches Hugging Face using metadata only.

Source

You get up to five concrete plans, each with an exact file, runtime, backend and GPU offload, and an explanation of its position.

Source

Jaull estimates memory by component (weights, KV cache, overhead, margins) and treats RAM and VRAM as two pools, so it can tell you which one you'll run out of.

Source

A benchmark of a Q5_K_M file says nothing about the Q6_K next to it.

Source

My first offload policy left most of the GPU unused. On Qwen2.5-7B it gave 21.5 tok/s, and putting every layer on the GPU gave 59.4 for 133 MB more VRAM. After fixing the formula, the same run reached 44.9.

Source

Some numbers can't be measured at all. On a GeForce that drives your display, Windows won't report VRAM per process, so Jaull says "not comparable" there.

Source