Make Pass Rate a Quorumed Histogram Before Late Workers Flip the Gate

说明与用途

问题：批量评估中的可变通过率可能被延迟或重复分数改变。解决方案：使用不可变直方图日志、时代封锁、工件哈希，并仅在达到法定人数且无在飞时代时关闭直方图。

问题

A batch evaluator stored pass rate as a mutable scalar, allowing late or stale worker scores to alter the gate decision, leading to unstable model releases.

结果

The new protocol prevents late or stale scores from altering the pass rate after the gate opens, ensuring stable model releases and correct quorum enforcement.

工作流

• 1. Issue a lease with (run_id, epoch, digest) to a worker.
• 2. Worker hashes artifact before scoring and sends score with epoch and digest.
• 3. Coordinator appends score to histogram log if epoch matches fenced epoch and digest matches.
• 4. Coordinator tracks accepted count and watermark; when accepted >= quorum and no in-flight epochs remain, close histogram.
• 5. After closure, any new score is rejected and logged.
• 6. Ship gate reads pass rate only from closed histogram, never from mutable scalar.

证据

The invariant is that a ship/no-ship pass rate is legal only when it is derived from a closed histogram of scores whose (run_id, artifact_digest) pairs are unique, whose worker epochs are still fenced in, and whose accepted count meets a declared quorum denominator.

来源

Append bins, never mutate a mean. The log stores counts per bin. The mean is a pure function of a closed snapshot.

来源