Why I Built a Vector Database Around Disk Instead of RAM

Описание и применение

AsterVec – это векторная база данных, хранящая индекс на диске, чтобы ограничить потребление RAM и повысить производительность при работе с LLM и RAG.

Задача

Embedded vector stores grow in RAM, competing with LLMs and other AI components on a single machine.

Результат

AsterVec delivers 5.2× the queries per second and 7.3× the inserts per second of Chroma at 100K vectors, using half the memory, and maintains comparable recall. At 1M vectors it serves from disk with ~0.3 GB RAM versus Chroma’s 1.2 GB, keeping query speed and recall similar while freeing memory for other AI components.

Рабочий процесс

• 1. Identify memory bottleneck in embedded vector store.
• 2. Split HNSW index into two on‑disk stores: graph edges in an LSM‑tree, vectors in a page‑based array with locality cache.
• 3. Keep only navigation layers and caches in RAM, bounding resident footprint.
• 4. Benchmark against Chroma and LanceDB on SIFT dataset at 100K and 1M vectors.
• 5. Deploy AsterVec via pip and integrate into local RAG pipeline.

Доказательства

AsterVec serves 5.2x Chroma's queries per second and 7.3x its inserts per second, in half the memory, with recall on par.

Источник

At a million vectors, AsterVec is genuinely serving from disk: Queries run in roughly 0.3 GB of engine memory against Chroma's 1.2 GB, query speed stays in the same ballpark, and recall is slightly ahead.

Источник