Multirun: A Multi-Agent Run Framework

Описание и применение

Многократный запуск кода агентом позволяет выявить вариативность модели, сравнить результаты и принять обоснованное решение, избегая ошибок, связанных с однократным выполнением.

Задача

Running a coding agent once gives only an anecdote; to understand model variability and avoid blind decisions, multiple isolated runs are needed.

Результат

Ten parallel runs revealed that all produced the same schema but split 5/5 on two design decisions and 8/2 on a NULL constraint, demonstrating model variability. The framework enabled cheap evaluation and informed branching decisions, saving potential costly mistakes.

Рабочий процесс

• 1. Define source repository at a specific SHA and optional database seed.
• 2. Create a task as a markdown prompt with checks, versioned as files.
• 3. Configure a fan‑out run: set global defaults and per‑run overrides (here all runs identical).
• 4. For each run, create a git worktree and a Postgres branch off the same parent.
• 5. Execute the agent (Claude Code via Anthropic Agent SDK) on the task in the isolated environment.
• 6. Capture artifacts (code, transcripts, diffs).
• 7. Diff each run’s branch back to the parent to obtain schema and data deltas.
• 8. Teardown the branch after diffing to avoid orphaned resources.
• 9. Store run state and artifacts in a Postgres results database for queryability.
• 10. Review the spread of outputs to inform evaluation or branching decisions.

Доказательства

Ten runs in this fan‑out took fifty minutes of wall time and about thirty‑two dollars at API rates

Источник

All ten reached the same schema shape. Then they split 5/5 on whether deleting a team deletes its data, 5/5 on the join table's primary key, and 8/2 on whether the new column is allowed to be NULL.

Источник

Ten runs need ten databases, and they all need to begin life identical.

Источник