Fine‑Tune an LLM on Your Mac with MLX: QLoRA in 16 Seconds (Real Run, No Cloud)

Description and use

On a MacBook Pro M2 Pro, a 3 B QLoRA adapter was trained in 16 seconds, reducing answer length from 151 to 47 tokens and speeding generation from 84 to 81 tokens/s, though the lowest‑loss checkpoint produced overly terse answers.

Problem

Fine‑tune a 3B language model on a Mac without cloud or NVIDIA hardware, achieving a short, clear answer style in under 20 seconds.

Outcome

A 3B QLoRA adapter was trained in ~16 seconds on a Mac, reducing answer length from 151 to 47 tokens and improving speed from 84 to 81 tokens/s, but the lowest‑loss checkpoint produced overly terse answers.

Workflow

• Install MLX LM with uv pip install "mlx-lm[train]"
• Convert the 4‑bit Qwen2.5‑3B‑Instruct weights with mlx_lm.convert --hf-path ./models/qwen2.5-3b-instruct -q --q-bits 4 --mlx-path ./models/Qwen2.5-3B-Instruct-4bit
• Create a JSONL training set with system, user, and assistant messages
• Run LoRA training: mlx_lm.lora --model ./models/Qwen2.5-3B-Instruct-4bit --train --data ./data --adapter-path ./adapters/smoke --batch-size 1 --num-layers 8 --max-seq-length 512 --learning-rate 1e-5 --iters 20 --grad-checkpoint --mask-prompt
• Evaluate loss on unseen examples with mlx_lm.lora --model ... --test --data ./data --adapter-path ./adapters/smoke
• Fuse the chosen adapter into a standalone model with mlx_lm.fuse --model ./models/Qwen2.5-3B-Instruct-4bit --adapter-path ./adapters/smoke-it10 --save-path ./models/rainy-qwen-3b
• Serve the fused model via FastAPI, loading the model once and using the same system prompt used during training

Evidence

The training took about 16 seconds, used about 2.5 GB of memory, and needed no cloud and no NVIDIA card. Just Apple's MLX.

Source

In this run that meant 3.3 million trainable parameters out of 3.09 billion: 0.108% of the model. Because the base model is 4-bit, this is QLoRA.

Source

Step 20 looks like the clear winner. Then I asked it the question: "Authentication verifies who you are."

Source