Video Generators as Candidate World Models › Physical Plausibility versus Visual Realism

The following probes separate visual appearance from properties a particular task may require. None is a universal pass/fail rule for every scene.

• Contact and intersection. Under a rigid-body assumption, rendered solids should not pass through one another when the relevant surfaces are visible.
• Motion under forces. Changes in velocity should agree with the modeled forces, contacts, friction, and camera motion; deceleration can have an ordinary cause.
• Occlusion and identity. Track an object through hidden intervals, allowing for uncertainty, and test whether reappearance is consistent with the earlier evidence.
• Object count. Check counts only in a controlled field of view where entry, exit, creation, and destruction rules are specified.
• Uncertainty calibration. Compare predictive samples with repeated reference outcomes, including both stochastic and near-deterministic transitions.
• Decision usefulness. Compare closed-loop task outcomes against appropriate policies and planners under the same observation, action, and compute budgets.

Some properties can be judged from a rendered view; others require an external tracker, simulator state, repeated trials, or a downstream control task. Contact is geometric, while motion tests require a stated dynamics model and forces. An attractive frame cannot substitute for those references.

These benchmarks ask different questions. IntPhys (Riochet et al., 2018) and Physion (Bear et al., 2021) test visual physical prediction or reasoning. Physics-IQ (2025) and VideoPhy (Bansal et al., 2024) directly evaluate physical behavior or commonsense in generated video and report shortcomings in the systems they tested. Their tasks, sampled models, and scoring procedures differ; neither establishes a universal ranking of video generators for control.

On the perceptual side, FVD (Unterthiner et al., 2018) compares feature statistics of sets of videos; frame FID, PSNR/SSIM, and LPIPS use different image-level comparisons. Their scores can be useful, but none isolates whether a generated object's motion follows the commanded action. Do not assume FVD is invariant to a displacement; test whether the chosen metric actually detects the task-critical change in the evaluation set.

For action-conditioned planning, include at least one task-relevant action-response test and, where the task demands it, a physical constraint test. Perceptual metrics supplement rather than replace those tests. A model that ignores actions can satisfy a static appearance constraint, so vary actions while holding the initial scene fixed.