NeuronDump: look inside any local LLM and make it smaller

说明与用途

NeuronDump 让本地 LLM 用户通过删除不必要的层、头和词元来检查和裁剪 GGUF 文件，生成仍能在 Ollama 或 llama.cpp 中加载的更小模型。

问题

Local LLM users download large GGUF files and treat them as black boxes, lacking insight into which model components are essential and which can be removed to reduce size without breaking functionality.

结果

NeuronDump enables users to identify and remove non‑essential parts of GGUF models, producing smaller files that still load in llama.cpp or Ollama and maintain comparable performance.

工作流

• 1. Install NeuronDump via prebuilt binaries or cargo.
• 2. Run neurondump list to discover local models.
• 3. Inspect a model with neurondump inspect <model> to view architecture, layers, heads, vocab, and template.
• 4. Use neurondump analyze <model> --out plan.json to generate a pruning plan.
• 5. Edit or review plan.json as needed.
• 6. Apply the plan with neurondump slim <model> --plan plan.json --only vocab --out small.gguf to create a trimmed GGUF.
• 7. (Optional) Build with --features trace to compare the cut model against the original in memory, evaluating token predictions, KL divergence, perplexity, and logit lens.

证据

I wanted to open that box: see the layers, the attention heads, the vocabulary, the chat template, and then ask a simple question: which parts does this model not actually need, and what breaks if I remove them?

来源

It reads GGUF files directly, with nothing hard-coded per model.

来源

Cut each layer (or head) in turn, rank them by damage, and write the least important ones out as a smaller GGUF plus an Ollama Modelfile.

来源