Improving Claude Code Success Rates with RE-call Memory Layer

Описание и применение

Эксперимент показал, что использование динамической памяти RE-call с Claude Code повышает успешность выполнения задач на 22.2% по сравнению со статическими файлами инструкций (CLAUDE.md). Статические файлы снизили эффективность до 36.1% против 50.0% у базовой модели, вероятно, из-за шума. RE-call обеспечивает точный поиск контекста, но увеличивает затраты на токены.

Задача

Static instruction files (like CLAUDE.md) can introduce noise or conflicting constraints, leading to lower success rates in deterministic coding tasks compared to bare models. Agents need a way to retrieve relevant project history without polluting the context window.

Результат

RE-call improved task success rates by 22.2 percentage points over static CLAUDE.md files and 8.3 points over bare Claude Code. The improvement was statistically significant (p=0.000145). Static files performed worse than bare models, likely due to noise. RE-call costs more per task but prevents expensive failure modes.

Рабочий процесс

• Set up a temporary repository for each task.
• Define deterministic checkers to validate the final state of the repository.
• Run tasks using three configurations: Bare Claude Code, Claude Code with CLAUDE.md, and Claude Code with RE-call memory.
• Verify memory tool availability before counting sessions.
• Compare success rates and analyze paired comparisons to determine statistical significance.
• Review cost and token usage for each configuration.

Доказательства

RE-call improved success over the CLAUDE.md baseline by 22.2 percentage points.

Источник

The paired McNemar test gave a p-value of 0.000145.

Источник

The static CLAUDE.md configuration performed worse than the bare configuration: Bare 50.0%, CLAUDE.md 36.1%.

Источник