800 Million Tokens for $36: Benchmarking DSH + DeepSeek V4 Pro on a Real Codebase

说明与用途

DeepSeek Harness 评测显示，LLM 能够在真实项目中持续工作，完成 45 次提交，处理 6.12 亿个 token，费用 28.35 美元，并实现了重要的架构、算法和形式化验证改进。

问题

Most coding‑agent benchmarks evaluate isolated fixes, not sustained, real‑world development.

结果

The agent performed 45 commits, processed 612 M tokens for $28.35, and produced substantial architectural changes, algorithmic improvements, and formal proofs, demonstrating sustained, iterative engineering behavior.

工作流

• Set up DSH with DeepSeek V4 Pro
• Ran the agent on the repo for several days
• Collected API usage and token metrics
• Reviewed commits and diffs
• Analyzed algorithmic changes and benchmarks
• Documented outcomes and lessons

证据

612,923,681 tokens processed for $28.35

来源

45 commits across 193 changed files

来源

Rewrite Natural storage around 64-bit limbs

来源