I built a local AI assistant fine‑tuned on my own conversations
Description and use
JERN is a local AI assistant fine‑tuned on personal conversations, built on a 1.5B Qwen model. It runs on the user’s machine, accesses local files and shell, and employs a tool loop for safe command execution.
Problem
Most assistants run on remote servers, ignore personal files, and sound generic. I needed a private, local assistant that understands my voice, can access my files and shell, and is small enough to run on a single GPU.
Outcome
JERN provides a fully local AI assistant that can understand the user’s conversational style, access local files, execute shell commands, and use a tool loop for safe interactions. It runs on a modest GPU and can also leverage a free Colab GPU via SSH tunnel.
Workflow
- 1. Clone the JERN repo from GitHub.
- 2. Install dependencies with pip.
- 3. Run
python main.pyto start the FastAPI server and web UI. - 4. The model downloads the base Qwen model and fine‑tunes it on a text file of personal conversations (takes minutes on GPU).
- 5. The assistant answers user queries, emits tool calls (files, shell, web search, etc.), the server executes them locally, and returns results to the model.
- 6. Optionally, run
python main.py colab --gpu=T4 --serveto use a free Colab GPU while keeping tools local via SSH tunnel.
Evidence
I wanted an AI assistant that is actually mine: it runs on my machine, knows how I talk, and can use my files and shell.
Everything runs on my machine with one command: python main.py opens the web UI at http://127.0.0.1:8000.
The training data is one text file, data/chat/conversations.txt, with [N] conversations I wrote: how JERN introduces itself, how it should talk, and examples of when to call a tool.
Built-in tools cover files, shell, web search, downloads and system info.
Because the tools are local, JERN can read my real files and run real commands. That is useful and also a risk, so review what it wants to run before you approve it.
- Record date: 2026-10-24T08:11:15.986Z