The World-Model Qualification Test › Summary

• A world model is qualified not by prediction loss alone but by four criteria: state sufficiency, action conditioning, counterfactual validity, and decision usefulness.
• State sufficiency is a property of the representation; its signature is irreducible error that added history cures. Belief states and recurrent latents are the standard remedies.
• Action conditioning must be verified interventionally, because logged data entangles actions with the behavior policy that produced them.
• Counterfactual validity is relative to a declared region of state-action space. In this experiment, changing the behavior policy inflated off-policy error by roughly two orders of magnitude while in-policy scores remained small.
• Closed-loop evidence is the final decision test: compounding drift, model exploitation, and policy-induced distribution shift can be invisible to one-step validation and consequential inside an agent.
• In the worked experiment, extra history improved prediction and control only modestly, while feedback let both learned controllers beat the blind baseline. The qualification battery rejected stronger conclusions that the original hypothesis invited.

The natural next question is what kinds of world models exist and where each sits along the axes this chapter exposed: observable versus latent states, fixed versus learned dynamics, prediction-centric versus decision-centric objectives. The next chapter, Taxonomies of World Models, takes up that classification.