Enforcing LLM Procedural Compliance via Typed Schemas

Description and use

Using strict typed schemas (TypeScript) to enforce procedural constraints on LLMs. Instead of free-form text, the model must provide structured data with justification for every item, improving audit reliability.

Problem

Frontier LLMs fail to follow complex procedural constraints (e.g., reviewing every item in a list) when using free-form Chain of Thought, resulting in low compliance rates (e.g., 9.91%) and hidden omissions.

Outcome

By enforcing procedural compliance through typed schemas rather than relying on free-form Chain of Thought, the workflow ensures that LLMs explicitly address every item in a dataset. This shifts the guarantee from result correctness to procedural adherence, making omissions visible and invalid.

Workflow

• Define the procedural requirements as a strict TypeScript interface (e.g., IAutoBeInterfaceEndpointReviewApplication).
• Structure the schema to require explicit decisions for every input item (e.g., keep, create, update, erase).
• Mandate a 'reason' field for every decision to force explicit justification.
• Use a validator to reject any submission that misses items, duplicates entries, or lacks reasons.
• Iterate on the schema by backtesting against historical cases to identify coverage gaps.
• Treat the schema as the enforcement mechanism, not just the prompt.

Evidence

9.91% is not 'did the model get it right on the first try' — it's 'did the model walk through the procedure to the end.'

Source

Prompt is a request. Schema is enforcement. Backtesting is what matures the schema.

Source