Postmortem: Why Our One-Shot AST Sanitizer for LLM Code Collapsed Under Real-World Constraints

说明与用途

后记说明了为什么一次性 AST 清理器在真实环境下崩溃：隐藏的初始化延迟、深层 AST 解析导致的计算爆炸以及信号处理器中的不安全操作。

问题

A one-shot AST sanitizer designed to intercept and sanitize LLM-generated code failed under realistic workloads, causing catastrophic timeouts and CPU exhaustion.

结果

The sanitizer’s design caused hidden latency from initialization, computational blowup in unparse on deep ASTs, and unsafe signal handling, leading to frequent timeouts and crashes in real deployments.

工作流

• 1. Receive JSON payload via sys.stdin.buffer
• 2. Validate schema with Pydantic v2
• 3. Parse code string with ast.parse
• 4. Transform AST with ast.NodeTransformer to replace prohibited calls
• 5. Fix missing locations with ast.fix_missing_locations
• 6. Unparse AST back to source with ast.unparse
• 7. Serialize result with orjson and write to stdout

证据

The timer was armed at the topmost entrypoint of the script: ... signal.alarm(TIMEOUT_SECONDS)

来源

In microbenchmarks on an idle developer machine, interpreter initialization took under 40 milliseconds. But inside CI/CD containers, heavily loaded Kubernetes worker nodes, or environments subject to cgroup CPU throttling, the initialization path accumulated massive hidden latency

来源

When ast.NodeTransformer replaces target nodes, it breaks location metadata (lineno, col_offset), requiring a full reconciliation pass via ast.fix_missing_locations() prior to code generation

来源