Implementing Conversational Search with Manticore and LLMs

说明与用途

本文介绍了如何使用 Manticore Search 和 LLM 实现对话式搜索。传统搜索仅返回列表，缺乏解释能力。提出的解决方案使用混合搜索进行检索，并利用 LLM 基于上下文生成答案，同时保持对话历史。Manticore 在对话式搜索质量基准测试中取得了最佳成绩。

问题

Traditional full-text and vector search return lists of results but fail to handle complex, multi-turn user queries that require explanation, comparison, or refinement of results based on context.

结果

Manticore achieved top scores on a conversational search quality benchmark (Hit@3: 0.3650, Hit@5: 0.4250, Hit@10: 0.5250, MRR: 0.2790) using 200 deterministic shopping queries. The system successfully links LLM answers to specific product sources, ensuring verifiability.

工作流

• Define a Conversational Search model using CREATE CHAT MODEL, specifying the LLM provider (e.g., OpenRouter), timeout, retrieval limit, max document length, and a system prompt enforcing context-only answers.
• Execute the initial query using CALL CHAT, providing the user query, target table, model name, conversation UUID, and vector field.
• The system uses conversation history to generate a complete search query, performs vector search, retrieves documents, and passes them to the LLM.
• The LLM generates an answer based on retrieved context, citing source IDs.
• Handle follow-up queries by calling CALL CHAT with the same conversation UUID, allowing the system to use history to resolve ambiguous references.
• Access the same functionality via the JSON API by sending a chat object to the /search endpoint.

证据

In the current run, Manticore scored 0.3650 Hit@3, 0.4250 Hit@5, 0.5250 Hit@10, and 0.2790 MRR — the best result on each of those metrics among the engines tested.

来源

Google reported in May 2026 that AI Mode had surpassed one billion monthly users.

来源