POCKET-35B-GGUF: Running a 35-Billion-Parameter LLM on CPU Without a GPU

Description and use

POCKET‑35B‑GGUF enables running a 35‑billion‑parameter LLM on CPU without GPU, using GGUF format and llama.cpp.

Problem

Deploying a 35‑billion‑parameter LLM on consumer or on‑premise CPU hardware without GPU access.

Outcome

The workflow enables local inference of a 35B LLM on CPU, expanding accessibility for developers and organizations lacking GPU resources.

Workflow

• Download the GGUF model via Hugging Face CLI: huggingface-cli download VIDRAFT/POCKET-35B-GGUF --local-dir ./pocket-35b
• Run inference with llama.cpp CLI: ./llama-cli -m ./pocket-35b/<model-file>.gguf -p "Explain transformer attention in one paragraph."
• Optionally start an OpenAI‑compatible server: ./llama-server -m ./pocket-35b/<model-file>.gguf --port 8080

Evidence

VIDRAFT's POCKET-35B-GGUF is a 35-billion-parameter large language model quantized and packaged in GGUF format for CPU-only inference — no GPU required.

Source

The practical tradeoff is latency: CPU inference at 35B scale is slower than GPU inference, but for many use cases — document analysis, batch summarization, local coding assistance, air-gapped environments — throughput-per-dollar or data-sovereignty requirements make this a worthwhile exchange.

Source

The Mindverse source article focuses on the model's availability and CPU-inference positioning rather than publishing specific benchmark scores.

Source