Claude Code’s Max Effort Cost Me 8x More. It Wrote the Same Code

Описание и применение

Уровни усилий в Claude Code увеличивают стоимость, время и токены, но не улучшают корректность кода. Они создают более защитный и модульный код, но без повышения качества тестов.

Задача

Assess whether higher effort settings in Claude Code improve code quality, correctness, or efficiency for software development tasks.

Результат

Higher effort settings in Claude Code increase cost, time, and output tokens but do not improve correctness or test pass rates. They produce more defensive, factored code and self-checks, which may be desirable in some contexts but add review overhead.

Рабочий процесс

• 1. Set up a headless Claude Code process per effort level, each in its own project copy. 2. Run the same prompt for each level, capturing stream-json logs. 3. Parse logs for turns, tool calls, cost, tokens, and hidden test results. 4. Repeat for three distinct tasks (Python implementation, Go port, bug hunt without execution).
• 5. Compare metrics: turns, wall time, cost, output tokens, hidden test pass rates, code quality checks (gofmt, go vet).
• 6. Analyze code differences manually to assess defensive coding and spec fidelity.
• 7. Summarize findings on effort impact.
• 8. Draw conclusions on when effort settings matter.
• 9. Document cost and time trade-offs.
• 10. Provide actionable guidance on using effort settings effectively.

Доказательства

Every level got the same score on every task. Only the clock and the invoice moved.

Источник

Higher effort spent its budget on self-checks and defensive code.

Источник

The total bill for this experiment was about $20.

Источник