A tool that tells you which local LLM your PC can actually run

Описание и применение

Jaull – это инструмент, который определяет, какие локальные модели языка можно запустить на конкретном ПК, учитывая CPU, RAM, GPU и доступные бэкенды. Он предлагает до пяти конкретных планов с точными файлами, временем выполнения и настройками GPU‑offload, а также предоставляет измеренные показатели скорости.

Задача

Determine which local language models a PC can run given its hardware constraints.

Результат

Jaull accurately predicts which LLMs fit on a given PC, provides benchmarked speed metrics, and highlights GPU offload inefficiencies.

Рабочий процесс

• Detect CPU, RAM, GPU and available backends
• Ask user questions about usage, priorities, languages, commercial intent
• Search Hugging Face metadata for compatible models
• Estimate memory by component (weights, KV cache, overhead, margins) treating RAM and VRAM as separate pools
• Generate up to five concrete plans with file, runtime, backend, GPU offload
• Provide explanations of plan positions
• Allow user to benchmark or score any plan on their machine
• Save results with SHA-256, machine, and protocol for future reordering

Доказательства

Jaull is an open-source Python tool with a terminal UI. It detects your CPU, RAM, GPU and backends, asks a few plain questions (what you'll use it for, what matters most, languages, commercial use) and searches Hugging Face using metadata only.

Источник

You get up to five concrete plans, each with an exact file, runtime, backend and GPU offload, and an explanation of its position.

Источник

Jaull estimates memory by component (weights, KV cache, overhead, margins) and treats RAM and VRAM as two pools, so it can tell you which one you'll run out of.

Источник

A benchmark of a Q5_K_M file says nothing about the Q6_K next to it.

Источник

My first offload policy left most of the GPU unused. On Qwen2.5-7B it gave 21.5 tok/s, and putting every layer on the GPU gave 59.4 for 133 MB more VRAM. After fixing the formula, the same run reached 44.9.

Источник

Some numbers can't be measured at all. On a GeForce that drives your display, Windows won't report VRAM per process, so Jaull says "not comparable" there.

Источник