Pin the Envelope, Not the Model Name

说明与用途

使用三种摘要的三重检测器可发现免费/试用端点中的隐藏模型更改，防止评估分数回归。

问题

Regression from routing changes in free/trial endpoints can silently alter evaluation scores.

结果

Tripwire detects silent model swaps by comparing request, envelope, and stamp digests, preventing unnoticed regression in evaluation scores.

工作流

• 1. Generate request fingerprint of the prompt and parameters.
• 2. Compute envelope digest of the response text, finish reason, and usage keys.
• 3. Record declared stamp (endpoint identity).
• 4. Store all three digests per run in a JSON line file.
• 5. On subsequent runs, compare request, envelope, and stamp digests to baseline.
• 6. If request digest unchanged but envelope digest differs and stamp unchanged, flag a potential silent model swap.
• 7. Use paired sign test to assess statistical significance of score changes.

证据

Every call should produce three independent digests, because each one answers a different question about who changed what.

来源

Write one JSON line per case per run, and never overwrite the file.

来源