What interactive video world models actually do on one GPU (and a tool to check)

Description and use

LiveFrame measures real frame rates of interactive video world models on any GPU, providing a verifiable ledger and revealing gaps between claimed and actual performance.

Problem

Publishers quote frame rates for interactive video world models without specifying hardware, leading to misleading expectations for users who run models on single GPUs.

Outcome

LiveFrame measured three published models on a single GPU, revealing significant discrepancies between claimed and actual fps. The tool’s design ensures low‑bias timing, verifiable receipts, and CPU‑first synthetic testing, enabling users to assess real‑world performance and cost per concurrent stream.

Workflow

• Install LiveFrame via pip
• Run a synthetic CPU benchmark to prove the loop end‑to‑end
• Execute a real model run on the target GPU
• Verify the run’s ledger to ensure integrity
• Optionally export results to worldmark/wbench formats

Evidence

LiveFrame measures what a model actually delivers on the hardware you run it on, and binds every number to a digest ledger anyone can re‑verify.

Source

Matrix‑Game 2.0 25 fps (1× H100) measured 19.7 fps – same H100, consistent with the claim.

Source

LiveFrame times frames as they're delivered to the host and counts native decoded frames only — never interpolated or duplicated ones.

Source

Every run writes a self‑contained report.html, a machine‑readable report.json, and a digest‑bound ledger.json. liveframe world verify <run>/ledger.json re‑checks every artifact byte.

Source