A ghost in my shell, powered by a local Gemma

Description and use

A terminal ghost uses a local Gemma model to generate contextual comments on shell commands without sending data outside localhost.

Problem

Shell commands expose sensitive data; a local LLM can generate contextual comments without leaking history.

Outcome

The ghost provides timely, model‑generated commentary on shell commands with minimal latency (≈2 s for gemma3:4b) and no external data transfer.

Workflow

• Install Ollama and pull gemma3:4b
• Clone haunted‑terminal repo
• Add haunt.sh to ~/.bashrc
• Set optional env vars (HAUNT_ODDS, HAUNT_MODEL, HAUNT_HOST)
• Run commands; ghost whispers appear after command completion

Evidence

It is about 80 lines: a Bash hook (haunt.sh) on PROMPT_COMMAND and a stdlib-only Python script (whisper.py) that calls Ollama's /api/generate.

Source

Same prompt, 5 commands, 16 cores, no GPU, 14 GB RAM (warm = model already loaded):

Source

12B writes slightly better lines, but 4B is the right default.

Source

Cold start can silence the ghost.

Source