Generative Simulation Benchmarking for autonomous urban air mobility routing under real-time policy constraints

说明与用途

静态基准无法捕捉动态政策事件，导致误导性成功率。生成式堆栈结合风和政策事件模型实现了更真实的评估。

问题

Static benchmarks for UAM routing fail to capture dynamic policy events, leading to misleading success rates.

结果

Incorporating a generative policy layer revealed failure modes that static benchmarks missed, reducing the apparent success rate from 98% to 61% and highlighting the need for dynamic policy modeling.

工作流

• 1. Define the routing problem with physical, policy, and temporal constraints.
• 2. Build a three‑layer architecture: world model, policy process, agent evaluation.
• 3. Implement a VAR(1) wind field generator for fast, reproducible scenarios.
• 4. Implement a Hawkes‑style policy event process to generate dynamic TFRs, curfews, and altitude caps.
• 5. Run agentic harness to roll out episodes, score them, and adapt scenario difficulty.
• 6. Compare success rates before and after incorporating the generative policy layer.

证据

Routes that scored 98% success in static evaluation dropped to 61% — not because the agent was bad, but because my benchmark itself was blind to the distribution of policy events.

来源

I modeled TFRs as a spatiotemporal point process conditioned on city context (events, weather, time of day), my benchmark started surfacing failure modes I'd never have hand-authored.

来源

For the physical layer, I used a conditional diffusion model trained on historical wind and traffic traces to produce plausible futures. But for the benchmark’s scoring loop, I found a lighter-weight approach more practical: a vector-autoregressive process with learned residual noise.

来源