I Built a Bot That Reproduces GitHub Bugs Automatically — Here's What Broke (and What I Learned)

Описание и применение

Ghost Hunter автоматически воспроизводит ошибки из описаний GitHub‑issues, устраняя необходимость ручной настройки окружения и ускоряя отладку, но выявил риски безопасности, которые были быстро устранены.

Задача

Maintainers spend excessive time reproducing bugs from vague issue reports, delaying debugging and fixes.

Результат

Automated bug reproduction reduced manual effort, improved issue triage, and surfaced security concerns early, leading to prompt fixes.

Рабочий процесс

• Comment bot/reproduce on an issue
• Ghost Hunter parses the issue with an LLM to extract environment and steps
• Ghost Hunter spins up an isolated Docker container matching the environment
• Ghost Hunter runs the reproduction steps inside the sandbox
• Ghost Hunter posts crash logs back to the issue

Доказательства

Within the first day, two commenters independently flagged real security concerns I hadn't fully thought through: Prompt injection risk and Network isolation.

Источник

Metabase's own internal bug-reproduction tool deliberately requires a human-in-the-loop trigger specifically to prevent this exact class of attack on public repos.

Источник