Kaiju: an agent bake‑off for your repo

Description and use

Kaiju lets you benchmark multiple coding agents by running the same prompt in isolated worktrees, automatically testing and ranking the outputs, and merging the best result into your main branch.

Problem

Determine which coding agent best solves a task in a specific codebase by running the same prompt across multiple agents and comparing results.

Outcome

Kaiju enables side‑by‑side comparison of coding agents, automates testing and ranking, and promotes the best run, providing a reproducible bake‑off workflow.

Workflow

• Start Kaiju daemon
• Open dashboard or use CLI to submit a prompt and select agents
• Kaiju creates isolated git worktrees and tmux sessions for each agent
• Agents run the prompt and generate code changes
• Dashboard shows live diffs and status
• Press Judge to run a test command (e.g., cargo test) and rank runs
• Select winner and click Promote to merge changes into workspace
• Leaderboard records results for future reference

Evidence

Kaiju: a small Rust daemon that runs one coding task across several AI CLIs at once (Claude Code, Codex, Gemini), each in its own isolated git worktree, then lets my tests decide the winner and merges that run back into my branch.

Source

Hit Judge and a local claude -p ranks the runs. Two things keep it honest: The runs are anonymized as A/B/C, so the judge can't favor a brand.

Source

One click commits the winning run's changes and merges its branch into your workspace. It refuses if your workspace is dirty or the run is still active, and it aborts cleanly on a conflict and tells you which branch to merge by hand.

Source