I measured 8 weeks of Claude Code: 87% lower cost, 86% first-try typecheck

Описание и применение

Измерено 8 недель работы Claude Code: экономия 87 % от стоимости токенов благодаря кэшированию и 86 % успешных первых попыток проверки типов в TypeScript‑приложении.

Задача

Determine the cost savings from prompt caching and the first-try pass rate of typechecks and tests when using Claude Code in a large TypeScript project.

Результат

Prompt caching achieved 98.4% cache read, reducing cost by 87.4% over 7.5B input tokens. First-try typecheck passed 86.3% of the time, tests 75.2% (filtered). Unfiltered test pass rate dropped to 52.4%.

Рабочий процесс

• Collect Claude Code session logs from ~/.claude/projects/**/*.jsonl
• Parse assistant responses for token usage and tool calls
• Deduplicate responses by message id
• Calculate cache ratios and savings using token multipliers
• Replay sessions to identify first-try typecheck and test runs
• Count passes vs. attempts for each check type
• Report filtered and unfiltered pass rates
• Generate summary and actionable insights

Доказательства

Over 8 weeks and 7.5B input tokens: Input tokens read from cache 98.4%

Источник

Typecheck 205 attempts, 86.3% passed on first run

Источник

One gotcha: Claude Code logs one line per content block, all with the same message id. If you don't deduplicate by id, you count the same response several times.

Источник