git bisect run on 1,000 Commits: 10 Runs and 3 Wrong Answers

Description and use

Using git bisect run to locate a bug in a 1,000‑commit repo: 10 tests instead of 389, with false positives avoided by external test scripts and exit‑125 for untestable commits.

Problem

Finding a bug in a large commit history efficiently while avoiding false positives caused by missing test files or broken builds.

Outcome

Bisect run located the planted bug in 10 runs (0.45 s) versus 389 linear checks (17.8 s). Three false positives were avoided by moving the test outside the repo and using exit‑code 125 for untestable commits.

Workflow

• Plant bug at commit 613
• Write test that fails only for the bug
• Create external check script that runs test
• Run git bisect start bad good
• Run git bisect run ~/bisect/check.sh
• If broken commits exist, add exit 125 before test
• Optionally skip known bad ranges with git bisect skip
• Review bisect output for first bad commit
• Reset with git bisect reset

Evidence

git bisect run found a planted bug in 1,000 commits with 10 test runs, a linear walk needed 389

Source

A test file that only existed in newer commits made bisect blame commit 2 instead of 613

Source

Without exit code 125, 41 broken builds made it blame commit 560; with it, 17 runs found 613

Source

I tagged every commit c1 to c1000 so I could name them. Small side lesson: 53 of those 1,000 names were also valid short object hashes in that repo, and Git printed warning: refname 'c997' is ambiguous every time I used one.

Source