Enforcing LLM Procedural Compliance via Typed Schemas

Описание и применение

Использование строгих типовых схем (TypeScript) для обеспечения соблюдения процедурных ограничений LLM. Вместо свободного текста модель должна предоставлять структурированные данные с обоснованием для каждого элемента, что повышает надежность аудита.

Задача

Frontier LLMs fail to follow complex procedural constraints (e.g., reviewing every item in a list) when using free-form Chain of Thought, resulting in low compliance rates (e.g., 9.91%) and hidden omissions.

Результат

By enforcing procedural compliance through typed schemas rather than relying on free-form Chain of Thought, the workflow ensures that LLMs explicitly address every item in a dataset. This shifts the guarantee from result correctness to procedural adherence, making omissions visible and invalid.

Рабочий процесс

• Define the procedural requirements as a strict TypeScript interface (e.g., IAutoBeInterfaceEndpointReviewApplication).
• Structure the schema to require explicit decisions for every input item (e.g., keep, create, update, erase).
• Mandate a 'reason' field for every decision to force explicit justification.
• Use a validator to reject any submission that misses items, duplicates entries, or lacks reasons.
• Iterate on the schema by backtesting against historical cases to identify coverage gaps.
• Treat the schema as the enforcement mechanism, not just the prompt.

Доказательства

9.91% is not 'did the model get it right on the first try' — it's 'did the model walk through the procedure to the end.'

Источник

Prompt is a request. Schema is enforcement. Backtesting is what matures the schema.

Источник