Implementing Conversational Search with Manticore and LLMs

Описание и применение

Статья описывает реализацию разговорного поиска с помощью Manticore Search и LLM. Традиционный поиск возвращает списки, но не объясняет результаты. Предложенное решение использует гибридный поиск для извлечения данных и LLM для генерации ответов на основе контекста, поддерживая историю диалога. Manticore показал лучшие результаты в бенчмарке качества разговорного поиска.

Задача

Traditional full-text and vector search return lists of results but fail to handle complex, multi-turn user queries that require explanation, comparison, or refinement of results based on context.

Результат

Manticore achieved top scores on a conversational search quality benchmark (Hit@3: 0.3650, Hit@5: 0.4250, Hit@10: 0.5250, MRR: 0.2790) using 200 deterministic shopping queries. The system successfully links LLM answers to specific product sources, ensuring verifiability.

Рабочий процесс

• Define a Conversational Search model using CREATE CHAT MODEL, specifying the LLM provider (e.g., OpenRouter), timeout, retrieval limit, max document length, and a system prompt enforcing context-only answers.
• Execute the initial query using CALL CHAT, providing the user query, target table, model name, conversation UUID, and vector field.
• The system uses conversation history to generate a complete search query, performs vector search, retrieves documents, and passes them to the LLM.
• The LLM generates an answer based on retrieved context, citing source IDs.
• Handle follow-up queries by calling CALL CHAT with the same conversation UUID, allowing the system to use history to resolve ambiguous references.
• Access the same functionality via the JSON API by sending a chat object to the /search endpoint.

Доказательства

In the current run, Manticore scored 0.3650 Hit@3, 0.4250 Hit@5, 0.5250 Hit@10, and 0.2790 MRR — the best result on each of those metrics among the engines tested.

Источник

Google reported in May 2026 that AI Mode had surpassed one billion monthly users.

Источник