Closing the KNN rescoring gap between columnar and row-wise storage

说明与用途

在 Manticore Search 中，默认的 KNN 重新评分在列式存储下比行式存储慢。使用内存映射的列式访问将吞吐量差距降低到行式存储的 85–92%，同时保持了对超出内存的数据集的高效处理。

问题

Default KNN rescoring in Manticore Search makes columnar vector access slower than row-wise storage, creating a performance gap of 2.80x–3.53x in KNN throughput.

结果

Memory-mapped columnar access reduces the KNN rescoring gap from 2.80x–3.53x to 1.08x–1.15x, achieving 85–92% of row-wise throughput while maintaining out-of-core performance.

工作流

• Measure KNN throughput on DBpedia dataset with row-wise and columnar storage.
• Identify rescoring bottleneck caused by reusable buffer.
• Implement memory-mapped columnar access to provide stable pointers.
• Batch rescoring: sort candidates, collect pointers, prefetch data, calculate distances in batches.
• Re-run benchmarks to compare throughput.
• Repeat with out-of-core taxi dataset to evaluate performance under memory pressure.
• Compare cold and hot run times for file vs mmap access.

证据

With the previous columnar access path, that step was much slower than with row-wise storage. In our DBpedia benchmark, row-wise storage delivered 2.80x–3.53x the KNN throughput.

来源

We changed columnar access to use memory mapping. This gives vectors stable addresses while still allowing the operating system to load and evict file pages on demand. The result is 2.57x–3.13x higher KNN throughput, reaching 85–92% of row-wise performance.

来源