Why I Built a Vector Database Around Disk Instead of RAM

Description and use

AsterVec is an on‑disk vector database that splits the HNSW index into separate stores for vectors and graph edges, limiting RAM usage while delivering high query and insert throughput compared to embedded alternatives.

Problem

Embedded vector stores grow in RAM, competing with LLMs and other AI components on a single machine.

Outcome

AsterVec delivers 5.2× the queries per second and 7.3× the inserts per second of Chroma at 100K vectors, using half the memory, and maintains comparable recall. At 1M vectors it serves from disk with ~0.3 GB RAM versus Chroma’s 1.2 GB, keeping query speed and recall similar while freeing memory for other AI components.

Workflow

• 1. Identify memory bottleneck in embedded vector store.
• 2. Split HNSW index into two on‑disk stores: graph edges in an LSM‑tree, vectors in a page‑based array with locality cache.
• 3. Keep only navigation layers and caches in RAM, bounding resident footprint.
• 4. Benchmark against Chroma and LanceDB on SIFT dataset at 100K and 1M vectors.
• 5. Deploy AsterVec via pip and integrate into local RAG pipeline.

Evidence

AsterVec serves 5.2x Chroma's queries per second and 7.3x its inserts per second, in half the memory, with recall on par.

Source

At a million vectors, AsterVec is genuinely serving from disk: Queries run in roughly 0.3 GB of engine memory against Chroma's 1.2 GB, query speed stays in the same ballpark, and recall is slightly ahead.

Source