I Built a Self-Hosted RAG Stack That Runs Without Cloud AI APIs

说明与用途

LocalRAG 是一个完全本地化的 RAG 堆栈，支持上传文档、生成嵌入并在不调用云 API 的情况下回答问题，适用于机密和隔离网络环境。

问题

Teams need document question‑answering without sending data to cloud AI services.

结果

LocalRAG successfully indexed a fictional employee handbook into three chunks and answered a leave‑policy question with 15 days, displaying source passages and similarity scores.

工作流

• Upload PDF/DOCX/TXT/Markdown
• Parse and split into chunks
• Generate embeddings with local bge‑m3 via Ollama
• Store vectors in Qdrant
• Embed user question
• Retrieve relevant chunks
• Generate answer with local LLM (qwen2.5:3b) via Ollama
• Return answer with source references

证据

Many teams want AI to answer questions from internal documents, but they cannot send those documents to a cloud service.

来源

LocalRAG allows a user to upload documents and ask questions against them.

来源

The application does not call a cloud AI API at runtime.

来源

I tested the current release with: Docker Desktop 4.95.0, Ollama 0.11.3, Qdrant, bge-m3 for embeddings, qwen2.5:3b for generation.

来源

The system: Indexed it into three chunks. Received the question: “How many days of annual leave do full‑time employees receive?” Retrieved the relevant policy text. Returned “15 days.”

来源