I Built a Bot That Reproduces GitHub Bugs Automatically — Here's What Broke (and What I Learned)

Description and use

Ghost Hunter automatically reproduces bugs from GitHub issue descriptions, eliminating manual environment setup and speeding debugging, while uncovering and addressing security risks.

Problem

Maintainers spend excessive time reproducing bugs from vague issue reports, delaying debugging and fixes.

Outcome

Automated bug reproduction reduced manual effort, improved issue triage, and surfaced security concerns early, leading to prompt fixes.

Workflow

• Comment bot/reproduce on an issue
• Ghost Hunter parses the issue with an LLM to extract environment and steps
• Ghost Hunter spins up an isolated Docker container matching the environment
• Ghost Hunter runs the reproduction steps inside the sandbox
• Ghost Hunter posts crash logs back to the issue

Evidence

Within the first day, two commenters independently flagged real security concerns I hadn't fully thought through: Prompt injection risk and Network isolation.

Source

Metabase's own internal bug-reproduction tool deliberately requires a human-in-the-loop trigger specifically to prevent this exact class of attack on public repos.

Source