NeuronDump: look inside any local LLM and make it smaller

Description and use

NeuronDump lets local LLM users inspect and trim GGUF files by removing unnecessary layers, heads, and tokens, producing smaller models that still load in Ollama or llama.cpp.

Problem

Local LLM users download large GGUF files and treat them as black boxes, lacking insight into which model components are essential and which can be removed to reduce size without breaking functionality.

Outcome

NeuronDump enables users to identify and remove non‑essential parts of GGUF models, producing smaller files that still load in llama.cpp or Ollama and maintain comparable performance.

Workflow

• 1. Install NeuronDump via prebuilt binaries or cargo.
• 2. Run neurondump list to discover local models.
• 3. Inspect a model with neurondump inspect <model> to view architecture, layers, heads, vocab, and template.
• 4. Use neurondump analyze <model> --out plan.json to generate a pruning plan.
• 5. Edit or review plan.json as needed.
• 6. Apply the plan with neurondump slim <model> --plan plan.json --only vocab --out small.gguf to create a trimmed GGUF.
• 7. (Optional) Build with --features trace to compare the cut model against the original in memory, evaluating token predictions, KL divergence, perplexity, and logit lens.

Evidence

I wanted to open that box: see the layers, the attention heads, the vocabulary, the chat template, and then ask a simple question: which parts does this model not actually need, and what breaks if I remove them?

Source

It reads GGUF files directly, with nothing hard-coded per model.

Source

Cut each layer (or head) in turn, rank them by damage, and write the least important ones out as a smaller GGUF plus an Ollama Modelfile.

Source