Building Local Hybrid Search for AI Coding Agents with LanceDB & Tantivy

Description and use

The rag-mcp tool provides a 100% local, private hybrid search solution for AI coding agents. Benchmarks show 90.9% strict relevance and 100% partial relevance on mixed codebases, eliminating retrieval failures and reducing hallucination compared to pure vector search or brute-force context stuffing.

Problem

AI coding agents face a trilemma in context management: greedy file probing wastes tokens and latency, brute-force context stuffing causes quadratic attention costs and hallucinations, and naive vector search fails on exact symbol names and error codes.

Outcome

The rag-mcp tool provides a 100% local, private hybrid search solution for AI coding agents. Benchmarks show 90.9% strict relevance and 100% partial relevance on mixed codebases, eliminating retrieval failures and reducing hallucination compared to pure vector search or brute-force context stuffing.

Workflow

• Clone the rag-mcp repository and install dependencies using uv.
• Run the automated test suite to verify installation.
• Configure the MCP server in the coding agent's configuration file (e.g., ~/.claude.json for Claude Code, ~/.codex/config.toml for Codex, or Cursor settings).
• Specify the absolute path to the Python interpreter and server.py script in the agent's MCP configuration.
• The agent queries the local hybrid search engine, which uses Tantivy for lexical search and LanceDB for vector search.
• Results are merged using Reciprocal Rank Fusion (RRF) with k=60.
• Top candidates are reranked by a local Cross-Encoder (ms-marco-MiniLM-L-6-v2) to prune false positives.
• The final relevant context is returned to the agent for reasoning.

Evidence

On real-world software codebases, combining breadcrumb AST chunking with RRF achieved 100% at-least-partial evidence capture and 0% total retrieval failure.

Source

Pure dense embedding search fails spectacularly on exact symbol names, compiler error codes, and unique variable identifiers.

Source