I Used an AI Agent to Test an Open-Source TypeScript Tool and Found a Real Bug

说明与用途

AI 代理对开源 TypeScript 工具进行黑盒测试，发现搜索过滤器组合错误，并提供可重现的报告，维护者已修复。

问题

Testing an open‑source TypeScript AI‑agent toolkit manually is time‑consuming; a concise, reproducible bug report is needed.

结果

The AI agent discovered a bug where combined search filters returned incorrect results. The maintainer fixed the issue after receiving a concise, reproducible report.

工作流

• Define a strict black‑box phase so the AI cannot inspect source code before testing the public experience.
• Use an isolated project with synthetic data to avoid exposing sensitive information.
• Require exact evidence: record OS, runtime, package version, commands, exit codes, expected and actual results.
• Preserve failed attempts to provide useful setup feedback.
• Review every conclusion to separate bugs from documentation or design questions.
• Report one confirmed problem at a time for easier review and fixing.

证据

The result was a reproducible bug report that the maintainer confirmed and fixed.

来源

The AI agent was instructed to behave like a new external user.

来源