Enforcing LLM Procedural Compliance via Typed Schemas

说明与用途

使用严格的类型化模式（TypeScript）来强制 LLM 遵守程序约束。模型必须为每个项目提供带有理由的结构化数据，从而提高审计的可靠性。

问题

Frontier LLMs fail to follow complex procedural constraints (e.g., reviewing every item in a list) when using free-form Chain of Thought, resulting in low compliance rates (e.g., 9.91%) and hidden omissions.

结果

By enforcing procedural compliance through typed schemas rather than relying on free-form Chain of Thought, the workflow ensures that LLMs explicitly address every item in a dataset. This shifts the guarantee from result correctness to procedural adherence, making omissions visible and invalid.

工作流

• Define the procedural requirements as a strict TypeScript interface (e.g., IAutoBeInterfaceEndpointReviewApplication).
• Structure the schema to require explicit decisions for every input item (e.g., keep, create, update, erase).
• Mandate a 'reason' field for every decision to force explicit justification.
• Use a validator to reject any submission that misses items, duplicates entries, or lacks reasons.
• Iterate on the schema by backtesting against historical cases to identify coverage gaps.
• Treat the schema as the enforcement mechanism, not just the prompt.

证据

9.91% is not 'did the model get it right on the first try' — it's 'did the model walk through the procedure to the end.'

来源

Prompt is a request. Schema is enforcement. Backtesting is what matures the schema.

来源