I Built an AI That Checks Whether Digital Products Deliver What They Promise
Описание и применение
AI‑система PromiseDiff проверяет, соответствуют ли цифровые продукты заявленным характеристикам, классифицируя каждое утверждение как подтверждённое, опровергнутое, не подтверждённое или вне области. Она использует Gemma для семантической проверки и PyMuPDF для метаданных, обеспечивая отчёт в Markdown до публикации.
Задача
Digital‑product listings often promise specific content (e.g., page counts, file formats, worksheets) that the delivered ZIP or PDF does not match, leading to refunds and negative reviews.
Результат
In a demo with a 48‑page A4 Budget Planner PDF, PromiseDiff confirmed the page count, flagged the missing US Letter edition, and exported a Markdown report. A benchmark of ten synthetic PDFs achieved 90% exact‑label accuracy, 100% contradiction precision, and 75% supported recall.
Рабочий процесс
- Paste listing text into the UI
- Upload the PDF or ZIP of the product
- PromiseDiff parses claims with Gemma
- Python validates measurable claims (page count, file existence, page size)
- Gemma evaluates semantic claims against extracted text
- Evidence validation checks filename, page, and exact quote
- Results are displayed with evidence and exported as Markdown
Доказательства
PromiseDiff shows what the files can back up, what they contradict, and what they simply cannot prove.
missing evidence != contradiction
exact-label accuracy = correct labels / total cases = 9 / 10 = 90%
- Дата записи: 2026-10-13T21:55:54.161Z