LocalLens: private document Q&A that runs 100% on your machine

Description and use

LocalLens is a local Q&A tool for confidential documents that runs entirely on the user’s machine, uses open‑weight models, cites source sentences, and abstains when the answer is not present.

Problem

Freelancers, HR, and legal staff often need to review confidential documents but cannot upload them to external services due to confidentiality clauses, forcing manual reading or violating privacy.

Outcome

LocalLens provides instant, privacy‑preserving answers to document questions, cites source sentences, and abstains when information is missing, avoiding hallucinations.

Workflow

• 1. Install LocalLens and download the Qwen2.5‑0.5B‑Instruct model and llama.cpp binary. 2. Run python3 -m locallens ask -f <document> -q "<question>". 3. The tool splits the document into sentences, ranks them by keyword overlap, and feeds the top sentences to the model. 4. The model returns an answer with cited sentence indices. 5. Optionally use --show-context to view the cited sentences for verification.

Evidence

Last year a freelancer friend was handed a 34-page client contract and asked to "just flag anything weird" before signing. She couldn't paste it into ChatGPT — the contract literally had a confidentiality clause forbidding exactly that.

Source

LocalLens — a command-line tool where you point at a document, ask a question in plain English, and get an answer in seconds. The twist: the AI model, the retrieval, and the inference engine all run on your own machine.

Source

The tool splits the document into sentences, ranks them by keyword overlap, and feeds the top sentences to the model.

Source

The model returns an answer with cited sentence indices.

Source

When the answer isn't in the document, it says so instead of inventing one.

Source