A tool that tells you which local LLM your PC can actually run

说明与用途

Jaull 是一个工具，可根据 CPU、RAM、GPU 和可用后端确定哪些本地语言模型可以在特定 PC 上运行。它提供多达五个具体方案，包含精确文件、运行时间和 GPU‑offload 设置，并提供测量的速度指标。

问题

Determine which local language models a PC can run given its hardware constraints.

结果

Jaull accurately predicts which LLMs fit on a given PC, provides benchmarked speed metrics, and highlights GPU offload inefficiencies.

工作流

• Detect CPU, RAM, GPU and available backends
• Ask user questions about usage, priorities, languages, commercial intent
• Search Hugging Face metadata for compatible models
• Estimate memory by component (weights, KV cache, overhead, margins) treating RAM and VRAM as separate pools
• Generate up to five concrete plans with file, runtime, backend, GPU offload
• Provide explanations of plan positions
• Allow user to benchmark or score any plan on their machine
• Save results with SHA-256, machine, and protocol for future reordering

证据

Jaull is an open-source Python tool with a terminal UI. It detects your CPU, RAM, GPU and backends, asks a few plain questions (what you'll use it for, what matters most, languages, commercial use) and searches Hugging Face using metadata only.

来源

You get up to five concrete plans, each with an exact file, runtime, backend and GPU offload, and an explanation of its position.

来源

Jaull estimates memory by component (weights, KV cache, overhead, margins) and treats RAM and VRAM as two pools, so it can tell you which one you'll run out of.

来源

A benchmark of a Q5_K_M file says nothing about the Q6_K next to it.

来源

My first offload policy left most of the GPU unused. On Qwen2.5-7B it gave 21.5 tok/s, and putting every layer on the GPU gave 59.4 for 133 MB more VRAM. After fixing the formula, the same run reached 44.9.

来源

Some numbers can't be measured at all. On a GeForce that drives your display, Windows won't report VRAM per process, so Jaull says "not comparable" there.

来源