Right Deadline, Wrong Party: Our AI Checks Accepted 28 of 28 Wrong Answers

Description and use

Safety checks that only verify number and word presence fail to detect when a deadline is assigned to the wrong party or action. Adding these checks to the model did not improve outcomes.

Problem

AI safety checks that only verify that numbers and words appear in the source fail to detect when a deadline is assigned to the wrong party or action.

Outcome

The custom checks passed all 28 wrong-party and wrong-action answers because they only verify presence of numbers and words, not the correct assignment of parties or actions. Adding the checks to the model did not improve performance; the model remained strict but not accurate.

Workflow

• 1. Construct 14 groups of legal passages, each with a question and three answer variants.
• 2. Run gemma2:2b on each passage-question-answer triple, asking it to ACCEPT or REJECT.
• 3. Apply custom checks: (C) every number must appear in the source; (D) every sentence must share enough words with the source.
• 4. Record verdicts for model alone, checks alone, and model plus checks.
• 5. Compare results to evaluate effectiveness of checks.
• 6. Publish set, scorer, and replies publicly.

Evidence

Our checks alone 28 of 28 wrong answers accepted

Source

Adding our checks to the model changed none of the 42 verdicts

Source

The only thing that changes inside a group is who does what

Source