Don't Trust Anyone's Muse Wonder Benchmark. Race It Against Opus 5.5 on Your Repo
说明与用途
本文提供了在用户自己的仓库中使用相同提示和隔离工作树或 Hermes IDE 会话,对 Muse Wonder 与 Opus 5.5 进行并排比较的逐步配方。
问题
Meta’s published benchmarks for Muse Wonder and Claude Opus 5.5 are not directly comparable because they use different prompts, tools, and harnesses. Developers need a reproducible way to compare the two models on their own codebase.
结果
The workflow enables side‑by‑side comparison of Muse Wonder and Opus 5.5 on the same codebase, ensuring identical prompts, harnesses, and environment, thereby producing trustworthy benchmark results.
工作流
- Choose a failing test ticket and a prompt that fixes it.
- Export a Meta Model API key as MODEL_API_KEY.
- Create a wrapper script claude-muse that sets ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_MODEL and related env vars to muse-wonder-1.0 and execs claude.
- Launch Hermes IDE session A with Claude Code and Opus 5.5, paste the prompt and run.
- Launch Hermes IDE session B with Custom agent using claude-muse, paste the same prompt and run.
- (Optional) Use plain git worktrees: git worktree add -b bakeoff/opus ../bakeoff-opus and bakeoff/muse, then run claude and claude-muse in separate terminals.
- (Optional) Install Meta’s muse CLI and run it in a third session for Meta’s own harness.
证据
Meta launched Muse Wonder on October 12, 2026
Both cost $4 per million input tokens and $20 per million output
Meta’s footnote on its SWE-Marathon row calls that row "not a controlled common-harness comparison"
- 记录日期: 2026-10-12T21:55:53.357Z