State, Physics, Causality, and Memory Evaluation › Summary

This chapter developed four evaluation families for asking what a world model represents, and each was anchored to a decision about information budgets and metric denominators.

• State probing measures accessibility to a specified readout on a specified distribution. Different probe capacities and controls can expose fitting failures, but failed decoding does not prove absent information. Trajectory-level splits and training-only preprocessing prevent particular leakage routes. Good decoding does not establish causal necessity or downstream use.

• Object permanence and temporal consistency concern hidden trajectories, reappearance and identity, which need separate tests. Horizon and occluder-width sweeps require explicit contributing-sample counts. Privileged baselines must disclose their information; incomplete extrapolators are not automatically error bounds. Memory ablations test sensitivity to a manipulation, not isolated memory circuits.

• Physical tests require a named regime, units and tolerance. Energy-ratio agreement can coexist with wrong trajectories, as the frozen-state control shows. Regime mismatch and model-class misspecification can both cause residuals; the sign alone does not identify the cause. An identity comparison with simulator truth is not an integration-accuracy certification.

• Interventions, counterfactuals, and causal fidelity separate observational prediction, population interventions, and unit counterfactuals. Here a near-perfect observational fit still gives roughly twice the structural action slope. Shared disturbance cancels in noiseless paired differences; the independently noisy paired average estimates the effect approximately. Report the target, identification design, practical overlap, feasibility, training interventions, structural coupling, and simulator-to-world limits.

The thread connecting all four is the same: state what the model received, state what the baseline received, name the quantity the metric measures, and separate offline prediction quality from closed-loop decision usefulness. That last separation is where the next chapter begins.