What interactive video world models actually do on one GPU (and a tool to check)

Описание и применение

LiveFrame позволяет измерять реальную частоту кадров интерактивных моделей видео на любой GPU, предоставляя проверяемый леджер и демонстрируя расхождения между заявленными и фактическими показателями.

Задача

Publishers quote frame rates for interactive video world models without specifying hardware, leading to misleading expectations for users who run models on single GPUs.

Результат

LiveFrame measured three published models on a single GPU, revealing significant discrepancies between claimed and actual fps. The tool’s design ensures low‑bias timing, verifiable receipts, and CPU‑first synthetic testing, enabling users to assess real‑world performance and cost per concurrent stream.

Рабочий процесс

• Install LiveFrame via pip
• Run a synthetic CPU benchmark to prove the loop end‑to‑end
• Execute a real model run on the target GPU
• Verify the run’s ledger to ensure integrity
• Optionally export results to worldmark/wbench formats

Доказательства

LiveFrame measures what a model actually delivers on the hardware you run it on, and binds every number to a digest ledger anyone can re‑verify.

Источник

Matrix‑Game 2.0 25 fps (1× H100) measured 19.7 fps – same H100, consistent with the claim.

Источник

LiveFrame times frames as they're delivered to the host and counts native decoded frames only — never interpolated or duplicated ones.

Источник

Every run writes a self‑contained report.html, a machine‑readable report.json, and a digest‑bound ledger.json. liveframe world verify <run>/ledger.json re‑checks every artifact byte.

Источник