I ran someone else's vibe-coded app through four AIs at once. Here’s what they found in an hour

Description and use

A four‑AI pipeline uncovered three critical security flaws in a Flask app within an hour: a hard‑coded master key, insecure JWT verification, and a random login failure due to padding errors. This shows that external AI reviews can catch hidden issues missed by the original model.

Problem

AI‑generated code often contains hidden security vulnerabilities that are not detected by the model that wrote it.

Outcome

The four‑model pipeline found a hard‑coded master key, insecure JWT verification, and a padding bug that caused intermittent login failures in an hour, proving that external AI reviews can uncover hidden security flaws.

Workflow

• 1. Pull open‑source repo (Python/Flask task‑management app).
• 2. Run first AI model to generate a plan and catch obvious problems.
• 3. Run second AI model to perform deep analysis of logic, edge cases, and side effects.
• 4. Run third AI model to inspect boundaries, safety, and overlooked issues.
• 5. Run fourth AI model to consolidate findings, decide clean or rework, and produce a report.
• 6. Review report, fix identified issues, and optionally re‑run pipeline for confirmation.

Evidence

The first builds a plan and catches the crude stuff. The second, the mean one, digs deep — what breaks, where the side effects are, which edge cases.

Source

Finding one: a master key hardcoded right into the source

Source

The app hands a logged‑in user a digital pass (a JWT) and later recognizes them by it.

Source