A docs assistant a small business can run on its own laptop, built with open-weight AI

Описание и применение

Малый бизнес может использовать локальный ассистент для ответов на вопросы из внутренних документов, сохраняя конфиденциальность и избегая затрат на облачные сервисы.

Задача

Small businesses with technical products repeatedly answer the same support questions from internal manuals, but their staff manually search PDFs and wikis and cannot easily use closed AI services.

Результат

The assistant enables small businesses to automate support queries locally, preserving privacy, eliminating per‑token costs, and avoiding vendor lock‑in.

Рабочий процесс

• 1. Install the assistant on the customer’s machine or server.
• 2. Load the customer’s documents into the system.
• 3. The assistant indexes document chunks using the BAAI/bge-small-en-v1.5 embedding model via fastembed and stores vectors in PostgreSQL with pgvector.
• 4. When a user asks a question, the assistant performs a hybrid search (vector + keyword) to find relevant pages.
• 5. The open‑weight LLM (Qwen2.5 1.5B/3B 4‑bit served by mlx‑lm) generates an answer using only the retrieved pages, adding citations.
• 6. The system logs each request and displays answered/unanswered status on a dashboard.
• 7. The customer can update documents and rerun tests to measure performance.
• 8. The customer pays a one‑time setup fee and a small monthly fee for document updates.

Доказательства

PumpGuard assistant: a question-and-answer assistant that answers from a company's own documents (manuals, FAQs, error-code tables), cites the page it used, and says "I could not find that in the knowledge base." when the answer isn't there.

Источник

Each chunk of the knowledge base is turned into a vector with BAAI/bge-small-en-v1.5 through fastembed, and stored in PostgreSQL with pgvector.

Источник

I ran it with Qwen2.5 (0.5B, 1.5B and 3B, 4-bit) served by mlx-lm on my 16 GB Mac, and compared the sizes on the same questions.

Источник

The writer may answer only from the retrieved pages, every answer carries citations, a role filter keeps staff-only pages (error codes) away from customers, and an audit log records each request.

Источник