I Built a Self-Hosted RAG Stack That Runs Without Cloud AI APIs

Описание и применение

LocalRAG – это полностью локальный стек RAG, который позволяет загружать документы, генерировать эмбеддинги и отвечать на вопросы без обращения к облачным API, что делает его пригодным для конфиденциальных и изолированных сетей.

Задача

Teams need document question‑answering without sending data to cloud AI services.

Результат

LocalRAG successfully indexed a fictional employee handbook into three chunks and answered a leave‑policy question with 15 days, displaying source passages and similarity scores.

Рабочий процесс

• Upload PDF/DOCX/TXT/Markdown
• Parse and split into chunks
• Generate embeddings with local bge‑m3 via Ollama
• Store vectors in Qdrant
• Embed user question
• Retrieve relevant chunks
• Generate answer with local LLM (qwen2.5:3b) via Ollama
• Return answer with source references

Доказательства

Many teams want AI to answer questions from internal documents, but they cannot send those documents to a cloud service.

Источник

LocalRAG allows a user to upload documents and ask questions against them.

Источник

The application does not call a cloud AI API at runtime.

Источник

I tested the current release with: Docker Desktop 4.95.0, Ollama 0.11.3, Qdrant, bge-m3 for embeddings, qwen2.5:3b for generation.

Источник

The system: Indexed it into three chunks. Received the question: “How many days of annual leave do full‑time employees receive?” Retrieved the relevant policy text. Returned “15 days.”

Источник