The World-Model Qualification Test › Criterion 4: Decision Usefulness and Closed-Loop Evidence

The final criterion is the one that pays the bills. A qualified world model, deployed inside a planning or learning agent, must improve decisions. This is a strictly weaker requirement than universal accuracy, which is good news: a model can be quantitatively sloppy yet qualitatively adequate if its errors do not change the ranking of available actions. It is also a strictly harder property to measure, because it depends on the agent, the task, and the interaction between the two.

The phrase "errors do not change the ranking of available actions" is worth expanding, because it identifies the precise sense in which decision usefulness relaxes the accuracy requirement. A planner consumes the model through an argmax: it compares candidate action sequences and picks the best according to imagined costs. For the final decision to be correct, the model need only rank the truly best sequence above the alternatives; it may distort costs substantially in regions the planner never seriously considers. This is why crude models sometimes control systems admirably, and why chasing the last decimal of prediction accuracy can be pure waste. Conversely, the requirement is harder to measure than accuracy because ranking correctness is defined relative to a specific set of alternatives, a specific reward, and a specific planner, none of which are properties of the model alone. Change any of the three and yesterday's qualified model may silently become unqualified.

Why insist on real interaction rather than simulated evaluation against a held-out model? Because a model's errors are not neutral to the planner that uses them. Three pathologies can be invisible to one-step held-out evaluation and become decision-relevant when the model is placed in a loop.

• Compounding drift. Long-horizon imagination accumulates error, so plans that look good at imagination time execute badly. The severity depends on the feedback structure of the task: stabilizing tasks forgive drift, balancing tasks punish it.
• Exploitation. As described above, the optimizer searches for and amplifies the model's worst extrapolations. The better the optimizer, the faster it finds the bugs.
• Distribution shift induced by the policy. A successful agent visits states its own training data never contained, so the model is continuously queried off-distribution by construction. Improvement changes the data distribution, which changes the model's reliability, which changes the agent. This loop is why model-based reinforcement learning interleaves training and deployment rather than treating modeling as a preprocessing step.

The feedback-structure remark in the first bullet deserves illustration, because it predicts which applications tolerate sloppy models. Consider a drone holding altitude versus a rocket balancing on its thrust column. If the drone's model errs, the resulting position error feeds back through the controller, which corrects it, and the physical system itself acts as an error-correction mechanism. The rocket has no such forgiveness: a small attitude error grows without intervention, so any model error that delays correction is amplified by the plant. In stabilizing tasks the closed loop partially masks model deficiencies; in unstable tasks it mercilessly exposes them. When you read claims about model-based agents succeeding with surprisingly simple models, check the feedback structure of the task before crediting the model.

Closed-loop evidence also disciplines a subtler temptation: improving the model on metrics nobody consumes. If raising rollout fidelity by 30% leaves the agent's return unchanged, the fidelity gain was irrelevant to the decisions at hand, and effort belongs elsewhere. Conversely, a modest model paired with a feedback-tolerant planner can beat a precise model paired with a brittle one. The pairing is the object of study.

With all four criteria in hand, we can compress the qualification test into a table you can keep beside your keyboard, shown in Table qualification-battery.

Notice the ordering of the columns: each row moves from an abstract question, to the least expensive experiment that answers it reliably, to the characteristic way a model fails. That structure is deliberate. In practice you run the tests roughly top to bottom, because the cheap representational tests come first and the expensive closed-loop deployments come last, and a failure early in the battery saves you the cost of everything after it.

The rest of this chapter runs the full battery on a concrete pair of models.