The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State‑Space, and Hybrid Sequence Models

Description and use

The audit uncovered hidden causal leakage in autoregressive models, detecting defects in two public releases. A simple two‑pass, no‑gradient check localizes the leak and ensures model correctness.

Problem

Autoregressive models may silently leak future information, inflating metrics and compromising downstream use, yet existing audits only inspect attention masks and miss leaks in other computation paths.

Outcome

The audit detected all 192 injected leaks and localized them, while mask inspection missed all. Two public models (Zamba2‑1.2B and Nemotron‑H‑8B) exhibited causal leakage at their chunk boundaries.

Workflow

• Generate a random token sequence x1 of length T.
• Create a perturbed sequence x2 by changing token at position T‑1.
• Attach forward hooks to all layers to capture outputs.
• Run forward pass f(x1) with caching disabled.
• Run forward pass f(x2) with caching disabled.
• For each layer ℓ compute Δℓ = max|h1ℓ[0:T‑1] – h2ℓ[0:T‑1]| (exclude perturbed position).
• Find the first layer ℓ* where Δℓ > threshold τ; if none, report CLEAN, else report LEAK at ℓ*.
• Optionally perform ε‑sweep to distinguish real leaks from numerical noise.

Evidence

We built a check that fits on one page, ran it on eight public checkpoints, and found real defects in two of them.

Source

Mask inspection finding nothing is not a bug in mask inspection. It is the point: the leaks were not in the masks.

Source

Two forward passes. No gradients. Runs on a CPU. Fits on a page.

Source