Decision Firewall: what building an AI governance framework taught me

Description and use

Decision Firewall is an open‑source platform for AI action authorization, audit, and recovery from failed requests, applied to subscription refund processing. It provides durable state, rule verification, and audit, but adds latency and integration complexity.

Problem

Developers needed a framework to authorize AI actions, audit outcomes, and handle uncertain execution, especially for subscription refunds where duplicate charges and timing rules apply.

Outcome

Decision Firewall achieved zero contract‑violating effects and zero unresolved attempts after recovery in synthetic tests, while a minimal model‑directed arm produced 95 violations across 170 trials. However, the framework added latency (p95 67.89 ms vs 26.52 ms) and required library‑contract workarounds, reducing its integration appeal.

Workflow

• Interpret the request and classify it as a duplicate‑charge claim
• Authorize the action by verifying ledger entries and policy eligibility
• Dispatch the refund and reserve the balance, recording the idempotency key
• Recover unresolved attempts by reconciling with the payment adapter and updating the audit record

Evidence

A duplicate-charge claim needs two payment records. Cancellation eligibility depends on timing and usage. A prior refund reduces the amount still available.

The recorded Laya run used CUDA in FP32: 28.42 seconds to load and 47.45 ms median inference across 34 assessments, including the first.

Source

The minimal arm produced 95 violating effects across 170 request trials.

Source