Better Vector Search for Long Documents: Chunking Inside Manticore Search

Description and use

Manticore automatically splits long documents into chunks at insert time, boosting recall@5 from 55.1 % to 83.3 % and MRR, with modest increases in RAM and ingest time.

Problem

Embedding models with limited token windows truncate long documents, causing loss of content and poor search recall.

Outcome

Using Manticore’s chunk_strategy increased recall@5 from 55.1% to 83.3% and MRR from 0.44 to 0.70 on a 189‑page manual, with only ~2.5× RAM and ~4× ingest time overhead.

Workflow

• Create a table with a float_vector_array column and set chunk_strategy (e.g., 'sentence'), max_tokens, and overlap_tokens.
• Insert documents; Manticore splits each document into chunks, generates embeddings for each chunk, and stores them in the array column.
• Run vector queries; Manticore searches all chunks and returns the document once, reporting the nearest chunk distance.
• Optionally adjust chunking parameters (max_tokens, overlap_tokens, max_chunks) to tune performance and recall.
• Monitor recall@5 and MRR to validate improvements.

Evidence

Recall@5 went from 55.1% → 83.3% and MRR from 0.44 → 0.70, at ~2.5× the RAM and ~4× the ingest time.

Source