Ghost in the Machine: Privacy-First Local RAG Microservice with FastAPI

Описание и применение

Микросервис GhostInTheMachine реализует локальный RAG на FastAPI, обеспечивая мгновенный поиск по векторному индексу без передачи данных в облако.

Задача

Cloud‑based AI APIs introduce latency, costs, rate limits, vendor lock‑in, and privacy risks when sensitive code or documentation is sent to third‑party endpoints.

Результат

The microservice achieves sub‑millisecond retrieval speeds, retains full data control, and eliminates external API dependencies.

Рабочий процесс

• POST /ingest – tokenize, normalize, store text and metadata in memory
• POST /query – convert query to term vector, compute cosine similarity against stored vectors, return top‑k matches
• Aggregate retrieved contexts and synthesize response payload locally

Доказательства

Cloud-based AI APIs have simplified adding intelligence to applications, but they come with significant trade-offs: network latency, recurring subscription costs, rate limits, and vendor lock-in.

Источник

GhostInTheMachine RAG is an asynchronous Retrieval-Augmented Generation (RAG) microservice built with FastAPI and Python 3.13.

Источник

The microservice processes incoming knowledge requests through a lean, three-stage local pipeline: Document Ingestion, Vector Similarity Search, Context Synthesis.

Источник