I Built a YouTube Credibility Scanner. Its Worst Bug Was Accusing Honest Creators

Description and use

A developer built a YouTube credibility scanner that initially misidentified creators’ own links as affiliate funnels. By separating hard and soft signals, the tool’s accuracy improved, though false positives and dislike‑count estimation issues remain.

Problem

The initial affiliate detector flagged every outbound link in a YouTube description as a funnel, incorrectly accusing creators who linked to their own products.

Outcome

After the fix, 15 of 20 new AI‑related videos were flagged as "Be Careful" and 5 were deemed "Likely Legit", matching expected patterns of affiliate funnels versus legitimate product sales.

Workflow

• 1. Identify outbound links in the video description.
• 2. Classify each link as hard or soft signal based on patterns.
• 3. If hard signals present, add weighted hard signal score and a small boost from soft signals; otherwise add no score.
• 4. Aggregate per-comment sentiment scores instead of a single aggregate.
• 5. Apply a raised baseline to prevent premature flagging.
• 6. Output the final credibility report.

Evidence

My first affiliate detector treated every outbound link in a description as one signal.

Source

I split detection into two tiers: Hard signals: known affiliate network patterns, referral parameters, discount-code formats; Soft signals: shorteners, redirects, link density.

Source

After the fix, I ran the scanner on 20 new “make money with AI” videos, mostly dropshipping, digital products, and AI side hustles. 15 came back “Be Careful”, 5 came back “Likely Legit”.

Source