Identical or Similar? AI-Powered Product Matching with Amazon Bedrock and S3 Vectors

Описание и применение

Проблема: выявление дубликатов и похожих товаров в каталоге видеоигр, где названия различаются. Решение: гибридный пайплайн, сначала правила, затем LLM Amazon Bedrock для неоднозначных случаев, хранение в S3. Итог: уменьшение дублирования с контролируемыми затратами.

Задача

Detecting duplicate or similar products in a catalog where names differ but refer to the same item, while avoiding costly AI calls for straightforward cases.

Результат

The MVP demonstrated that a hybrid rule‑and‑AI approach can identify duplicates with low cost, reserving expensive LLM calls for truly ambiguous cases.

Рабочий процесс

• 1. Load Amazon Reviews 2023 dataset
• 2. Filter to 20 top franchises using regex
• 3. Apply deterministic rules (e.g., exact title match, platform, edition) to flag duplicates
• 4. For remaining ambiguous pairs, construct prompt with business criteria and send to Bedrock LLM
• 5. Parse LLM response to classify as duplicate, variant, or distinct
• 6. Store results in S3 vector store for real‑time lookup
• 7. Expose API for real‑time queries and batch updates

Доказательства

We first analyze the data and apply deterministic rules to the cases that can be solved simply. The language model is reserved for cases that require interpreting text, recognizing abbreviations or evaluating ambiguous matches.

Источник

For the MVP, I selected a subset of products based on the 20 best-selling video game franchises according to Wikipedia.

Источник