How much RAM you actually need to run AI locally

Description and use

Users often choose AI PCs based on price or specs without knowing the real limiting factor—memory—leading to underpowered setups. The article explains how to determine the required memory for a target model and context, accounting for shared CPU/GPU memory, and how to select the right configuration to avoid unnecessary costs.

Problem

Users often choose AI PCs based on price or specs without knowing the real limiting factor—memory—leading to underpowered setups.

Outcome

By focusing on memory rather than price or cores, users can select a machine that actually supports their desired model, reducing cost and ensuring functionality.

Workflow

• Identify the target model size (small, mid-size, large).
• Consult the sizing table to determine required memory at 4‑bit quantisation.
• Estimate additional memory needed for context (conversation, system prompt, code repo).
• Check the machine’s unified‑memory architecture to see how much is available after OS overhead.
• Select a configuration (24 GB, 64 GB, 128 GB) that meets the total memory requirement.
• Compare prices across configurations to find the most cost‑effective option.

Evidence

about 5 GB at 4‑bit quantisation

about 15–20 GB at 4‑bit, before you add context

about 35–40 GB at 4‑bit, plus context

24 GB gets tight as soon as the operating system takes its share

The current generation runs from $2,599 to $5,999