Generative Simulation Benchmarking for autonomous urban air mobility routing under real-time policy constraints

Описание и применение

Статические бенчмарки для автономного воздушного транспорта не учитывают динамические события политики, что приводит к неверным оценкам эффективности. Создан генеративный стек, включающий модели ветра и политических событий, позволяющий оценивать агента в реалистичных условиях.

Задача

Static benchmarks for UAM routing fail to capture dynamic policy events, leading to misleading success rates.

Результат

Incorporating a generative policy layer revealed failure modes that static benchmarks missed, reducing the apparent success rate from 98% to 61% and highlighting the need for dynamic policy modeling.

Рабочий процесс

• 1. Define the routing problem with physical, policy, and temporal constraints.
• 2. Build a three‑layer architecture: world model, policy process, agent evaluation.
• 3. Implement a VAR(1) wind field generator for fast, reproducible scenarios.
• 4. Implement a Hawkes‑style policy event process to generate dynamic TFRs, curfews, and altitude caps.
• 5. Run agentic harness to roll out episodes, score them, and adapt scenario difficulty.
• 6. Compare success rates before and after incorporating the generative policy layer.

Доказательства

Routes that scored 98% success in static evaluation dropped to 61% — not because the agent was bad, but because my benchmark itself was blind to the distribution of policy events.

Источник

I modeled TFRs as a spatiotemporal point process conditioned on city context (events, weather, time of day), my benchmark started surfacing failure modes I'd never have hand-authored.

Источник

For the physical layer, I used a conditional diffusion model trained on historical wind and traffic traces to produce plausible futures. But for the benchmark’s scoring loop, I found a lighter-weight approach more practical: a vector-autoregressive process with learned residual noise.

Источник