Postmortem: Why Our One-Shot AST Sanitizer for LLM Code Collapsed Under Real-World Constraints

Описание и применение

Постмортем показывает, почему однократный AST‑санитайзер для кода LLM не выдержал реальных нагрузок: скрытая задержка и таймаут, рост сложности при распарсивании глубоких деревьев, а также небезопасные операции в обработчике сигналов.

Задача

A one-shot AST sanitizer designed to intercept and sanitize LLM-generated code failed under realistic workloads, causing catastrophic timeouts and CPU exhaustion.

Результат

The sanitizer’s design caused hidden latency from initialization, computational blowup in unparse on deep ASTs, and unsafe signal handling, leading to frequent timeouts and crashes in real deployments.

Рабочий процесс

• 1. Receive JSON payload via sys.stdin.buffer
• 2. Validate schema with Pydantic v2
• 3. Parse code string with ast.parse
• 4. Transform AST with ast.NodeTransformer to replace prohibited calls
• 5. Fix missing locations with ast.fix_missing_locations
• 6. Unparse AST back to source with ast.unparse
• 7. Serialize result with orjson and write to stdout

Доказательства

The timer was armed at the topmost entrypoint of the script: ... signal.alarm(TIMEOUT_SECONDS)

Источник

In microbenchmarks on an idle developer machine, interpreter initialization took under 40 milliseconds. But inside CI/CD containers, heavily loaded Kubernetes worker nodes, or environments subject to cgroup CPU throttling, the initialization path accumulated massive hidden latency

Источник

When ast.NodeTransformer replaces target nodes, it breaks location metadata (lineno, col_offset), requiring a full reconciliation pass via ast.fix_missing_locations() prior to code generation

Источник