Make Pass Rate a Quorumed Histogram Before Late Workers Flip the Gate

Описание и применение

Проблема: динамический показатель прохода в оценке батчей может изменяться из‑за поздних или дублированных результатов. Решение: использовать неизменяемый гистограммный лог, фэнсинг по эпохам и хеширование артефактов, закрывать гистограмму только при достижении кворума и без живых эпох.

Задача

A batch evaluator stored pass rate as a mutable scalar, allowing late or stale worker scores to alter the gate decision, leading to unstable model releases.

Результат

The new protocol prevents late or stale scores from altering the pass rate after the gate opens, ensuring stable model releases and correct quorum enforcement.

Рабочий процесс

• 1. Issue a lease with (run_id, epoch, digest) to a worker.
• 2. Worker hashes artifact before scoring and sends score with epoch and digest.
• 3. Coordinator appends score to histogram log if epoch matches fenced epoch and digest matches.
• 4. Coordinator tracks accepted count and watermark; when accepted >= quorum and no in-flight epochs remain, close histogram.
• 5. After closure, any new score is rejected and logged.
• 6. Ship gate reads pass rate only from closed histogram, never from mutable scalar.

Доказательства

The invariant is that a ship/no-ship pass rate is legal only when it is derived from a closed histogram of scores whose (run_id, artifact_digest) pairs are unique, whose worker epochs are still fenced in, and whose accepted count meets a declared quorum denominator.

Источник

Append bins, never mutate a mean. The log stores counts per bin. The mean is a pure function of a closed snapshot.

Источник