I Used an AI Agent to Test an Open-Source TypeScript Tool and Found a Real Bug

Описание и применение

AI‑агент протестировал открытый TypeScript‑инструмент, обнаружил реальную ошибку в работе фильтров поиска и предоставил воспроизводимый отчёт, который был исправлен.

Задача

Testing an open‑source TypeScript AI‑agent toolkit manually is time‑consuming; a concise, reproducible bug report is needed.

Результат

The AI agent discovered a bug where combined search filters returned incorrect results. The maintainer fixed the issue after receiving a concise, reproducible report.

Рабочий процесс

• Define a strict black‑box phase so the AI cannot inspect source code before testing the public experience.
• Use an isolated project with synthetic data to avoid exposing sensitive information.
• Require exact evidence: record OS, runtime, package version, commands, exit codes, expected and actual results.
• Preserve failed attempts to provide useful setup feedback.
• Review every conclusion to separate bugs from documentation or design questions.
• Report one confirmed problem at a time for easier review and fixing.

Доказательства

The result was a reproducible bug report that the maintainer confirmed and fixed.

Источник

The AI agent was instructed to behave like a new external user.

Источник