Asserts Passed, RSS Doubled: An rusage Envelope for AI C++ Patches

说明与用途

AI 生成的补丁可能通过所有功能测试，但会悄悄增加内存使用。添加包含 RSS、CPU 和超时限制的 JSON 封装，使评估工具能够检测到这些回归。

问题

AI‑generated C++ patches can pass all functional tests while silently increasing memory usage, leading to production regressions that are hard to detect with standard test harnesses.

结果

The resource envelope turned a silent memory regression into a detectable failure, ensuring AI patches respect performance contracts.

工作流

• 1. Write functional tests and run them. 2. Add a resource envelope JSON alongside fixtures, specifying max_rss_kb, max_user_us, timeout_s. 3. Run the evaluation harness, which measures rusage and compares against envelope ceilings. 4. If any ceiling is exceeded, the patch is rejected. 5. Repeat until both functional and resource constraints are satisfied.

证据

An eval harness that only checks exit codes will score the heavy patch as a win.

来源

The envelope is JSON next to the fixtures. It records ceilings, not averages.

来源