A Government Lab Spent £1,000 and Two Weeks Using Frontier AI to Find Cloud Misconfigurations. The Same Findings Cost $0.

Описание и применение

Государственный лабораторный центр использовал frontier‑AI для обнаружения конфигурационных уязвимостей в AWS за £1 000 и две недели, но те же уязвимости можно найти бесплатно и мгновенно с помощью детерминированного анализатора Stave.

Задача

Determine whether frontier AI is cost‑effective for finding cloud misconfigurations compared to deterministic configuration evaluation.

Результат

Frontier AI found a 5‑step privilege‑escalation chain for £150–1,000 and two weeks, but the same chain was deterministically discoverable for $0 in seconds using Stave.

Рабочий процесс

• 1. Selected three approaches: source‑code analysis, agentic probing, human‑in‑the‑loop red‑teaming.
• 2. Ran each approach over two weeks, consuming £150–1,000 in LLM tokens.
• 3. Triaged findings, separating noise from actionable issues.
• 4. Compared results to deterministic configuration evaluation.
• 5. Concluded that deterministic evaluation is cheaper, faster, and reproducible for configuration‑based findings.
• 6. Recommended using AI for application‑logic bugs and creative attack surface exploration.
• 7. Suggested continuous deterministic evaluation for infrastructure misconfigurations.

Доказательства

"The cost was under £1,000 in LLM tokens. The most capable model found the critical chain for under £150."

Источник

"The 5-step attack chain was the exercise's most significant discovery."

Источник

"A deterministic configuration evaluator takes a snapshot of the cloud infrastructure, including the IAM policies, resource configurations, trust relationships, organizational policies to evaluate compound predicates against it."

Источник

"The cost difference is categorical. One approach consumes frontier model compute for every exercise. The other runs locally, offline, with no external dependencies, on every commit."

Источник