Post Mortem: Terminal Bench Challenge - WASM Renderer

说明与用途

自动化代理将本地测试通过率从64提升至1334，但渲染器仍不完整，揭示了五个问题检查和工作流程问题。

问题

Agents’ local test passes rose from 64 to 1,334 while the renderer remained broken, revealing that many tests passed for the wrong reasons.

结果

The renderer could render a basic triangle and pass many local tests, yet conformance remained low; five checks passed for the wrong reasons, exposing gaps in test design and workflow governance.

工作流

• 1. Split work by role: architecture mapping, sprint planning, coding, testing, and independent review.
• 2. Build core subsystems (state layer, shader compiler/interpreter, rasterizer) by Sprint 8.
• 3. Run local tests; passes climbed from 64 to 1,334 by Sprint 13.
• 4. Rebuild captured source fresh and run custom probes to verify rendering of a triangle and simple shaders.
• 5. Identify and fix bugs through code reviews (e.g., matrix stride, compressed‑format query, negative zero division).
• 6. Detect workflow issues (e.g., automated operator decisions, test bypassing public entry point).
• 7. Document findings and lessons in a post‑mortem.
• 8. Publish article summarizing results and five problematic checks.

证据

Our agents' tests went from 64 passes to 1,334 while the renderer stayed broken.

来源

In Favur's 296-hour attempt at a Terminal-Bench WebGL renderer, local test passes rose from 64 to 1,334 while the last internal conformance results stood at 13 of 672 WebGL1 cases and 20 of 2,598 WebGL2 cases.

来源

The renderer could draw a plain red triangle, and every stage behind it is real.

来源