Identical or Similar? AI-Powered Product Matching with Amazon Bedrock and S3 Vectors

Description and use

Problem: detecting duplicate and similar products in a video game catalog with varying names. Solution: a hybrid pipeline that applies deterministic rules first and then Amazon Bedrock LLMs for ambiguous cases, storing results in S3. Outcome: reduced duplication with controlled costs.

Problem

Detecting duplicate or similar products in a catalog where names differ but refer to the same item, while avoiding costly AI calls for straightforward cases.

Outcome

The MVP demonstrated that a hybrid rule‑and‑AI approach can identify duplicates with low cost, reserving expensive LLM calls for truly ambiguous cases.

Workflow

• 1. Load Amazon Reviews 2023 dataset
• 2. Filter to 20 top franchises using regex
• 3. Apply deterministic rules (e.g., exact title match, platform, edition) to flag duplicates
• 4. For remaining ambiguous pairs, construct prompt with business criteria and send to Bedrock LLM
• 5. Parse LLM response to classify as duplicate, variant, or distinct
• 6. Store results in S3 vector store for real‑time lookup
• 7. Expose API for real‑time queries and batch updates

Evidence

We first analyze the data and apply deterministic rules to the cases that can be solved simply. The language model is reserved for cases that require interpreting text, recognizing abbreviations or evaluating ambiguous matches.

Source

For the MVP, I selected a subset of products based on the 20 best-selling video game franchises according to Wikipedia.

Source