Show HN: Fusion Check — a zero-data law that predicts when merging LLMs is safe

Описание и применение

Fusion Check предсказывает, безопасно ли объединять два дообученных LLM, используя закон без данных, и помогает принимать решение о слиянии, повышая производительность при совместимости.

Задача

Blind weight averaging (soup) can dilute performance when merging LLMs trained on different tasks.

Результат

Fusion Check enables safe merging of fine-tuned LLMs, improving performance in compatible cases and preventing dilution in incompatible ones, with predictions validated analytically, formally, and empirically.

Рабочий процесс

• 1. Compute task kernel metrics (cosine of weight deltas and effective dimensionality).
• 2. Apply the zero-data law: check if cos² ≥ 1 − δ²·d_eff/(2(1-α)²).
• 3. If condition satisfied, compute optimal mixing coefficient α* = √γ₁/(√γ₁+√γ₂).
• 4. Decide MERGE, NO_MERGE, PRIORITIZED, REFUSE, or VERIFY based on the law’s output.
• 5. If MERGE, perform weight averaging with α*; otherwise, keep models separate or re-evaluate.

Доказательства

Proven three independent ways: Analytic derivation, Formal verification with z3 (28 theorems), Empirical validation (147 combinations, max deviation 1.13%, zero violations).

Источник

Real result: Two weak, diverse estimators of the same task (0.762/0.756) → merged soup 0.822 (+6–7 points).

Источник