Fine‑Tune an LLM on Your Mac with MLX: QLoRA in 16 Seconds (Real Run, No Cloud)

说明与用途

在 MacBook Pro M2 Pro 上，使用 16 秒训练了 3 B QLoRA 适配器，将答案长度从 151 个标记缩短到 47 个标记，并将生成速度从 84 tokens/s 提升到 81 tokens/s，但最低 loss 的检查点导致答案过短。

问题

Fine‑tune a 3B language model on a Mac without cloud or NVIDIA hardware, achieving a short, clear answer style in under 20 seconds.

结果

A 3B QLoRA adapter was trained in ~16 seconds on a Mac, reducing answer length from 151 to 47 tokens and improving speed from 84 to 81 tokens/s, but the lowest‑loss checkpoint produced overly terse answers.

工作流

• Install MLX LM with uv pip install "mlx-lm[train]"
• Convert the 4‑bit Qwen2.5‑3B‑Instruct weights with mlx_lm.convert --hf-path ./models/qwen2.5-3b-instruct -q --q-bits 4 --mlx-path ./models/Qwen2.5-3B-Instruct-4bit
• Create a JSONL training set with system, user, and assistant messages
• Run LoRA training: mlx_lm.lora --model ./models/Qwen2.5-3B-Instruct-4bit --train --data ./data --adapter-path ./adapters/smoke --batch-size 1 --num-layers 8 --max-seq-length 512 --learning-rate 1e-5 --iters 20 --grad-checkpoint --mask-prompt
• Evaluate loss on unseen examples with mlx_lm.lora --model ... --test --data ./data --adapter-path ./adapters/smoke
• Fuse the chosen adapter into a standalone model with mlx_lm.fuse --model ./models/Qwen2.5-3B-Instruct-4bit --adapter-path ./adapters/smoke-it10 --save-path ./models/rainy-qwen-3b
• Serve the fused model via FastAPI, loading the model once and using the same system prompt used during training

证据

The training took about 16 seconds, used about 2.5 GB of memory, and needed no cloud and no NVIDIA card. Just Apple's MLX.

来源

In this run that meant 3.3 million trainable parameters out of 3.09 billion: 0.108% of the model. Because the base model is 4-bit, this is QLoRA.

来源

Step 20 looks like the clear winner. Then I asked it the question: "Authentication verifies who you are."

来源