Loading page...Загружаем страницу...正在加载页面...
AtlasRepo

Don't Trust Anyone's Muse Wonder Benchmark. Race It Against Opus 5.5 on Your Repo

说明与用途

本文提供了在用户自己的仓库中使用相同提示和隔离工作树或 Hermes IDE 会话,对 Muse Wonder 与 Opus 5.5 进行并排比较的逐步配方。

问题

Meta’s published benchmarks for Muse Wonder and Claude Opus 5.5 are not directly comparable because they use different prompts, tools, and harnesses. Developers need a reproducible way to compare the two models on their own codebase.

结果

The workflow enables side‑by‑side comparison of Muse Wonder and Opus 5.5 on the same codebase, ensuring identical prompts, harnesses, and environment, thereby producing trustworthy benchmark results.

工作流

  • Choose a failing test ticket and a prompt that fixes it.
  • Export a Meta Model API key as MODEL_API_KEY.
  • Create a wrapper script claude-muse that sets ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, ANTHROPIC_MODEL and related env vars to muse-wonder-1.0 and execs claude.
  • Launch Hermes IDE session A with Claude Code and Opus 5.5, paste the prompt and run.
  • Launch Hermes IDE session B with Custom agent using claude-muse, paste the same prompt and run.
  • (Optional) Use plain git worktrees: git worktree add -b bakeoff/opus ../bakeoff-opus and bakeoff/muse, then run claude and claude-muse in separate terminals.
  • (Optional) Install Meta’s muse CLI and run it in a third session for Meta’s own harness.

证据

Meta launched Muse Wonder on October 12, 2026

来源

Both cost $4 per million input tokens and $20 per million output

来源

Meta’s footnote on its SWE-Marathon row calls that row "not a controlled common-harness comparison"

来源

  • 记录日期: 2026-10-12T21:55:53.357Z