Blacksmith's GitHub Actions runners finished the same job 3 to 4 times faster than GitHub hosted Actions, across 10 trials

Description and use

Blacksmith runners finished the same job 3–4 times faster than GitHub-hosted Actions, but require an organization and may auto‑create PRs.

Problem

Benchmarking Blacksmith runners against GitHub-hosted ubuntu-latest to evaluate performance and identify operational constraints.

Outcome

Blacksmith runners completed the same Node.js scripts 3.17x to 3.77x faster than GitHub-hosted ubuntu-latest, with tighter variance and no overlapping ranges.

Workflow

• Move repo to a GitHub organization
• Install Blacksmith GitHub App on the organization
• Verify runner image availability
• Modify workflow runs-on to Blacksmith image
• Run benchmark workflow with 10 trials per job
• Collect timing data using date +%s%N
• Analyze mean, range, and speedup

Evidence

Blacksmith ran 3.17x to 3.77x faster with noticeably tighter variance

Source

Blacksmith refuses to run on a personal repository at all

Source

The Migration Wizard pull request opened automatically after app install

Source