The Mask Is Not the Model: Auditing Prefix Invariance in Attention, State‑Space, and Hybrid Sequence Models

Описание и применение

Модельный аудит выявил скрытое утечку информации в автогрессивных моделях, обнаружив дефекты в двух публичных релизах. Простая проверка, требующая только двух проходов без градиентов, позволяет локализовать утечку и гарантировать корректность модели.

Задача

Autoregressive models may silently leak future information, inflating metrics and compromising downstream use, yet existing audits only inspect attention masks and miss leaks in other computation paths.

Результат

The audit detected all 192 injected leaks and localized them, while mask inspection missed all. Two public models (Zamba2‑1.2B and Nemotron‑H‑8B) exhibited causal leakage at their chunk boundaries.

Рабочий процесс

• Generate a random token sequence x1 of length T.
• Create a perturbed sequence x2 by changing token at position T‑1.
• Attach forward hooks to all layers to capture outputs.
• Run forward pass f(x1) with caching disabled.
• Run forward pass f(x2) with caching disabled.
• For each layer ℓ compute Δℓ = max|h1ℓ[0:T‑1] – h2ℓ[0:T‑1]| (exclude perturbed position).
• Find the first layer ℓ* where Δℓ > threshold τ; if none, report CLEAN, else report LEAK at ℓ*.
• Optionally perform ε‑sweep to distinguish real leaks from numerical noise.

Доказательства

We built a check that fits on one page, ran it on eight public checkpoints, and found real defects in two of them.

Источник

Mask inspection finding nothing is not a bug in mask inspection. It is the point: the leaks were not in the masks.

Источник

Two forward passes. No gradients. Runs on a CPU. Fits on a page.

Источник