The World-Model Qualification Test

Suppose you are handed two trained models and told that each one achieves an impressive prediction score on a robotics dataset. Model A watches a camera stream and produces startlingly realistic videos of the next few seconds. Model B takes the same camera stream plus the robot's commanded action and predicts the next frame, less beautifully, but conditioned on what the robot does. Which of these is a world model?

If your answer is "both," consider what happens when we plug each one into a controller. Model A will happily hallucinate futures for any action you feed it, because nothing in its training taught it that actions cause outcomes. Model B can be asked a question a planner needs answered: "if the gripper closes now, where will the object be?" Only one of these models can sit inside a planning loop and change the agent's behavior for the better.

This thought experiment is worth pausing on, because it exposes something uncomfortable about how machine learning models are usually judged. Both models would look excellent in a paper or a leaderboard entry. Both produce low loss values, and both generate plausible-looking outputs. Yet they belong to entirely different categories of artifact. Model A is a forecaster: it extrapolates patterns in observation sequences. Model B is a candidate world model: it encodes a relationship between decisions and their consequences. The distinction only becomes visible when we ask what the model will be used for, and this is precisely the question that standard evaluation pipelines never ask. A video-generation score cannot tell you whether closing the gripper moves the object, because that information simply is not present in an unconditional prediction task.

This chapter gives you a practical instrument for telling them apart: the world-model qualification test. It is not a benchmark with a leaderboard. It is a set of four questions you ask of any candidate model, backed by experiments you can run. The questions are:

• State sufficiency: Does the model's internal state carry everything needed to predict the future?
• Action conditioning: Does the model treat actions as causes, not as decoration?
• Counterfactual validity: Do predictions hold for actions and situations the training data never contained?
• Decision usefulness: Does an agent that plans with this model do better?

Each criterion exists because a specific, common failure mode slips past naive evaluation. We will meet each failure mode, formalize the criterion that catches it, and then run all four tests on two competing models of the same physical system. By the end you will have a reusable experimental template alongside the vocabulary.

It is worth setting expectations about how these four criteria relate to one another before we dive in. They are ordered deliberately. The first two concern what the model represents internally, which is where most failures originate. The third concerns where those representations remain trustworthy, which is a property of the interaction between the model and its training data. The fourth concerns whether any of the preceding properties translate into better behavior, which is the only outcome that ultimately matters to the system's owner. Each criterion presupposes the ones before it: a model with an insufficient state cannot be action-conditioned in any meaningful sense, and a model that fails counterfactual validity cannot be trusted by a planner no matter how good its in-distribution scores look. Reading them as a pipeline, rather than as four independent checkboxes, is the mental habit this chapter tries to install.

In @ch-what-is-a-world-model we established the cast of characters: the environment evolves through transitions, the agent receives observations, chooses actions, and a world model is the agent's internal machinery for predicting how observations and rewards respond to actions. Everything in this chapter builds on that separation. The single most important consequence of it is this: the data an agent collects is generated by its own behavior, but the decisions it must make later live in situations its behavior creates. Evaluation procedures that ignore this gap systematically flatter bad models.

This circularity deserves a moment of careful attention, because it has no analogue in supervised learning on static datasets. When you train an image classifier, the test set is drawn from the same distribution as the training set by construction, and the classifier's predictions do not influence which images appear next. In sequential decision making, both guarantees vanish. The data distribution is a function of the behavior policy, and the deployment distribution is a function of the learned policy, so the two distributions are coupled through the very thing being learned. A model can be excellent at describing the world as the data-collection policy experienced it and simultaneously useless for describing the world as a new policy will create it. Every evaluation protocol that samples test cases from the logging distribution quietly assumes away this coupling, and thereby assumes away the hardest part of the problem.

The gap shows up in three distinct ways.

First, one-step error hides compounding. Most models are trained to predict one step ahead, where each prediction is graded against a ground-truth previous state. When the model is used for planning, however, its own outputs become its inputs. A 1% one-step error can become a useless prediction fifty steps later, because errors feed back into the model and grow. We saw the skeleton of this argument in Chapter 1 when our first world model drifted off course during long rollouts; here we will measure it precisely. The mechanism is easy to state but easy to underestimate: at each step of an imagined rollout, the model receives an input that is slightly wrong, produces an output that is therefore slightly more wrong, and passes that output forward. Under mild conditions the error grows geometrically rather than linearly, which means that horizon, not one-step accuracy, is often the binding constraint on usefulness. A planner that looks twelve steps ahead cares about a quantity that a one-step validation loss says nothing about.

Second, training distributions are not decision distributions. A model trained on logs of a careful human driver has seen gentle steering at moderate speeds. Ask it what happens during an emergency swerve and it is extrapolating. The model's accuracy on the logging distribution tells you almost nothing about its accuracy on the counterfactual actions a planner wants to explore. This is not a corner-case pathology; it is structural. Planners are, almost by definition, devices for finding actions that the behavior policy did not take, because if the best action were always the habitual one there would be nothing to plan. So the queries a planner directs at its world model are systematically concentrated exactly where the training data is thinnest. The better the optimizer inside the planner, the more aggressively it probes those thin regions, which is why improving the search algorithm can paradoxically make a model-based agent worse.

Third, a good predictor can still be a bad advisor. Prediction quality is measured against the truth; decision quality is measured against a reward. A model can be accurate everywhere the agent will go and still mislead the planner if its errors are largest exactly where the promising actions are, or if it is confident precisely where it should not be. To make this concrete, imagine two models with identical average error. One makes small uniform mistakes everywhere; the other is nearly perfect except that it systematically underestimates the cost of the single most attractive action sequence. Average error cannot distinguish them, but the second model is far more dangerous, because planners do not average over actions, they select the argmax. Decision quality depends on the error profile near the optimum, not the error profile overall, and these can be wildly different quantities.

These three gaps motivate the four criteria. State sufficiency and action conditioning concern what the model represents. Counterfactual validity concerns where its predictions hold. Decision usefulness concerns whether any of it matters when the model is placed inside an agent. Let us take them one at a time.

A world model predicts the future from an internal state. The first qualification question is whether that state contains enough information. Formally, we say a state representation is sufficient when the future is conditionally independent of the past given the state and the planned action:

where:

• $o_{t+1:T}$: the sequence of future observations from time $t+1$ up to horizon $T$
• $r_{t+1:T}$: the sequence of future rewards over the same horizon
• $s_t$: the model's internal state at time $t$
• $a_{t:T}$: the planned actions from time $t$ up to the horizon
• $o_{\le t}$: the full history of observations up to and including time $t$
• $a_{<t}$: the historical actions taken before time $t$

That is, once you know $s_t$ and the future actions, the raw history $o_{\le t}$ adds nothing. This is the predictive core of the Markov assumption, and it is a property of the representation rather than of the environment alone: a fully observable environment can still be modeled inadequately if the model compresses away a relevant variable.

The last clause is the one students most often miss, so let us dwell on it. Markovianity is usually introduced as a property of environments: either the physical system's current state determines its future or it does not. But the qualification question here is about the model's internal state, which is a lossy summary chosen by whoever designed or trained the model. Two models of the same perfectly Markovian cart can differ sharply in sufficiency if one keeps velocity information and the other discards it. Sufficiency therefore depends jointly on the environment and the representation: the environment determines what information exists, and the representation determines how much of it survives compression. When we say a model "has" a sufficient state, we mean the compression preserved everything the future depends on, which is an empirical claim about the trained artifact, not a design intention.

Why does this deserve its own criterion rather than being folded into "accuracy"? Because insufficiency produces a characteristic signature that accuracy numbers obscure: the model's residual error is irreducible from the outside. If a model of a car tracks only position and not velocity, no amount of training improves its prediction of where the car will be in one second, because the missing variable matters. The error looks like noise. It is structure the representation discarded.

Confusing discarded structure with irreducible noise is an expensive mistake in applied world modeling. Sensor noise is unpredictable given the available inputs. Insufficient-state error can look similar from the outside: residual variance plateaus, the loss curve flattens, and ordinary training signals say "training complete." Changing the input representation can reduce insufficient-state error but cannot remove sensor noise. Varying the input history is therefore a useful diagnostic: a lower plateau shows that the shorter representation discarded predictive structure. An unchanged plateau is weaker evidence. Optimization failure, limited model capacity, and finite data must also be ruled out before calling the remainder a noise floor.

Two common remedies anchor large portions of this book.

• Carry more history. Feed the model a window of past observations, or a recurrent summary of all past observations, so that latent quantities such as velocities can be inferred. Recurrent state-space models do exactly this, as we will see in Part V: World-Model Architectures.
• Learn a belief state. Maintain an explicit posterior over the hidden physical state given the observation history. This is the classical filtering approach, covered in detail in the Bayesian filtering chapter of Part II: Inference, Decisions, and Control.

These remedies embody different choices about where inference should happen. Carrying history delegates the work to the transition model: the network must extract velocity-like features from raw observation sequences. A belief state makes inference explicit: a filtering step maintains a posterior over hidden variables, and the dynamics operate on that posterior. History-based models often trade interpretability for flexibility; explicit filters trade stronger modeling assumptions for structured uncertainty. State observers and Kalman-style filters became central tools in modern control engineering, while many deep world models favor learned recurrent summaries. Part II and Part V develop both lineages in depth.

How do you test for sufficiency in practice? The cheapest diagnostic is incremental: train the same architecture while conditioning on progressively more history, and watch whether one-step error keeps falling. If adding the last three observations sharply reduces error but adding the last ten changes nothing, the model has extracted what history can offer. If error falls monotonically forever, the model is still starving. A sharper diagnostic is probing: train a small auxiliary regressor to predict known-relevant hidden quantities (velocity, contact state, the position of an occluded object) from the model's internal state. If the probe fails, the information is gone and no downstream improvement will recover it. We will develop these diagnostics properly in Part XI: Evaluation and Understanding.

Both diagnostics reward a little interpretation. The incremental-history diagnostic is essentially a nonparametric estimate of how much predictive information remains in the observation history beyond what the model already uses, so a flat curve is evidence that the architecture's input window is not the bottleneck. The probe diagnostic is stronger but requires domain knowledge: you must know which hidden quantities matter before you can check whether the state encodes them. In practice the two compose well. Use the history sweep to establish that more input would help or not, then use targeted probes to identify which latent variables are missing, since knowing that something is missing is less actionable than knowing what it is. A probe that recovers velocity from a recurrent state but not contact state tells you precisely which physical phenomenon your representation has failed to capture, and that diagnosis points directly at the remedy.

The second question separates world models from forecasters. A forecaster maps past observations to future observations. A world model maps past observations and a proposed action to future observations. The action must enter as a cause.

This sounds trivial, and architecturally it often is: concatenate the action vector onto the model input. But several subtle failures hide behind a nominally action-conditioned interface.

• The ignored-action model. The network receives the action but learns to rely on it so little that its predictions are nearly identical for all actions. This happens routinely when training data comes from a smooth feedback controller, because in such data the action is almost predictable from the state, and the easiest way to minimize loss is to predict the continuation of the state trajectory. The model passes every open-loop test on the logging distribution and is useless for planning.
• The spurious-correlation model. The model latches onto correlations between actions and outcomes that hold in the dataset but are not causal. Human drivers press the brake when the car ahead is close, so a naive model can learn "brake pressed $\to$ obstacle was near" and infer the wrong direction of influence. We will return to this confusability when we discuss causal world models in Part IV: Space, Physics, and Causality.
• The wrong-granularity model. The model conditions on high-level commands ("turn left") when the downstream planner issues low-level controls (wheel torques), or vice versa. The action interface must match the decision interface of the agent that will consume the model.

Each of these failures deserves a brief anatomy, because they fail for different reasons and are caught by different tests. The ignored-action model is a failure of data: the behavior policy made actions statistically redundant with states, so gradient descent had no incentive to use the action channel. It is caught by intervening on the action input and observing whether predictions change. The spurious-correlation model is a failure of causal identification: the model found real correlations and assigned them the wrong direction or mechanism. It is caught by testing predictions against interventions that break the correlation structure of the logging policy. The wrong-granularity model is a failure of interface design: the model is fine at its own level of abstraction but cannot answer the questions the consumer asks. It is caught not by statistics at all but by an integration review comparing the planner's action space against the model's conditioning space. Note that none of these three is reliably caught by held-out prediction loss on logged data, which is exactly why action conditioning gets its own criterion.

The deep reason this criterion is separate from accuracy is the difference between observational and interventional distributions. Accuracy on logged data measures $p(o_{t+1} \mid o_{\le t}, a_t)$ under the data-collecting policy. Planning requires the response function $p(o_{t+1} \mid o_{\le t}, \mathrm{do}(a_t))$: what would happen if we chose $a_t$, possibly contrary to habit. They coincide when the conditioned history blocks every common cause of action and outcome and the relevant actions have data support. Logged policies rarely guarantee both conditions. Data collection is therefore inseparable from qualification.

If the $\mathrm{do}(\cdot)$ notation is new, here is the intuition in plain language. Conditioning answers "given that you observe action $a_t$ was taken, what typically follows?" Intervention answers "if you force action $a_t$ to happen regardless of circumstances, what follows?" These differ whenever the observed action carried information about the situation, which it almost always does: a braking driver is usually braking because something is ahead. The observational query entangles the effect of the brake with the reason the brake was pressed. Only an intervention, or data collected under randomized actions, severs that entanglement. This is the same distinction that powers randomized controlled trials in medicine, transplanted into the training of predictive models, and it explains why exploration is a precondition for causal world modeling rather than a reinforcement-learning nicety.

Action conditioning asks whether the model responds to actions. Counterfactual validity asks whether those responses remain trustworthy away from the training distribution, which is exactly where a curious planner spends its time.

Three practices make this criterion concrete.

Evaluate on off-policy splits. Hold out data collected under a different behavior policy than training used, and report error separately for on-policy and off-policy slices. A model whose off-policy error explodes has learned the logging policy, not the world.

Probe the action margins. Systematically vary the action input around states the agent cares about and inspect the predicted response surface. When the system has a known monotone actuation effect over the probed range, the learned response should have the correct sign and a plausible gain. Unexpected reversals, discontinuities, or extreme slopes are evidence against the model.

Track uncertainty, not just point predictions. A model that knows where its data ends can warn the planner. A model that extrapolates confidently converts ignorance into confidently wrong plans. Uncertainty-aware world models are a first-class topic in Part XII: Reliable World Models; for qualification purposes it is enough to demand that the model have some notion of applicability, even a crude one.

A few words on why each practice works. Off-policy splits are the cheapest honest instrument because they require no new data collection, only disciplined bookkeeping: tag every trajectory with the policy that produced it, and refuse to report a single pooled number. The action-margin probe is a sanity check on the model's internal physics rather than its fit: derivative signs and monotonicity are among the few things we know about real systems independently of data, so violations are unambiguous evidence of learned nonsense. The uncertainty requirement is subtler than the other two, because calibration itself is hard to verify, but even a crude applicability signal changes the planner's failure mode from silent confident error to detectable out-of-envelope queries, and that difference is often the difference between a recoverable mistake and a catastrophic one.

Counterfactual validity interacts dangerously with compounding. During a long imagined rollout, a planner explores ever-further corners of the model's state space, and small counterfactual errors redirect the search into regions where errors are larger still. This is the mechanism behind model exploitation: the planner finds actions that look brilliant under the model and are merely exploits of its extrapolation bugs. We will study defenses against exploitation, such as pessimism and ensembles, in the model-based reinforcement learning chapter of Part II.

It helps to picture exploitation as an adversarial game between two components of your own system. The planner is an optimizer: give it a flawed model and enough search budget, and it can discover regions where the model departs from reality. The model is often least reliable in precisely those poorly explored regions. Neither component need be broken; their combination is. Many defenses limit the optimizer's trust or exposure: ensembles reveal disagreement, pessimistic objectives penalize uncertainty, support penalties discourage unsupported searches, and shorter horizons bound how long errors can compound. We will return to this pattern throughout the book.

The final criterion is the one that pays the bills. A qualified world model, deployed inside a planning or learning agent, must improve decisions. This is a strictly weaker requirement than universal accuracy, which is good news: a model can be quantitatively sloppy yet qualitatively adequate if its errors do not change the ranking of available actions. It is also a strictly harder property to measure, because it depends on the agent, the task, and the interaction between the two.

The phrase "errors do not change the ranking of available actions" is worth expanding, because it identifies the precise sense in which decision usefulness relaxes the accuracy requirement. A planner consumes the model through an argmax: it compares candidate action sequences and picks the best according to imagined costs. For the final decision to be correct, the model need only rank the truly best sequence above the alternatives; it may distort costs substantially in regions the planner never seriously considers. This is why crude models sometimes control systems admirably, and why chasing the last decimal of prediction accuracy can be pure waste. Conversely, the requirement is harder to measure than accuracy because ranking correctness is defined relative to a specific set of alternatives, a specific reward, and a specific planner, none of which are properties of the model alone. Change any of the three and yesterday's qualified model may silently become unqualified.

Why insist on real interaction rather than simulated evaluation against a held-out model? Because a model's errors are not neutral to the planner that uses them. Three pathologies can be invisible to one-step held-out evaluation and become decision-relevant when the model is placed in a loop.

• Compounding drift. Long-horizon imagination accumulates error, so plans that look good at imagination time execute badly. The severity depends on the feedback structure of the task: stabilizing tasks forgive drift, balancing tasks punish it.
• Exploitation. As described above, the optimizer searches for and amplifies the model's worst extrapolations. The better the optimizer, the faster it finds the bugs.
• Distribution shift induced by the policy. A successful agent visits states its own training data never contained, so the model is continuously queried off-distribution by construction. Improvement changes the data distribution, which changes the model's reliability, which changes the agent. This loop is why model-based reinforcement learning interleaves training and deployment rather than treating modeling as a preprocessing step.

The feedback-structure remark in the first bullet deserves illustration, because it predicts which applications tolerate sloppy models. Consider a drone holding altitude versus a rocket balancing on its thrust column. If the drone's model errs, the resulting position error feeds back through the controller, which corrects it, and the physical system itself acts as an error-correction mechanism. The rocket has no such forgiveness: a small attitude error grows without intervention, so any model error that delays correction is amplified by the plant. In stabilizing tasks the closed loop partially masks model deficiencies; in unstable tasks it mercilessly exposes them. When you read claims about model-based agents succeeding with surprisingly simple models, check the feedback structure of the task before crediting the model.

Closed-loop evidence also disciplines a subtler temptation: improving the model on metrics nobody consumes. If raising rollout fidelity by 30% leaves the agent's return unchanged, the fidelity gain was irrelevant to the decisions at hand, and effort belongs elsewhere. Conversely, a modest model paired with a feedback-tolerant planner can beat a precise model paired with a brittle one. The pairing is the object of study.

With all four criteria in hand, we can compress the qualification test into a table you can keep beside your keyboard, shown in Table qualification-battery.

Notice the ordering of the columns: each row moves from an abstract question, to the least expensive experiment that answers it reliably, to the characteristic way a model fails. That structure is deliberate. In practice you run the tests roughly top to bottom, because the cheap representational tests come first and the expensive closed-loop deployments come last, and a failure early in the battery saves you the cost of everything after it.

The rest of this chapter runs the full battery on a concrete pair of models.

We will build a deliberately simple physical system so that every number is checkable by hand: a damped cart moving in one dimension under a force input, observed through a noisy position sensor. The true state is the pair (position, velocity), but the sensor reports only position. This single design choice creates the entire drama: a model that looks only at the latest observation is missing velocity, which is exactly the quantity that determines where the cart goes next.

Simplicity here is a feature, not a concession. Because the true dynamics are known analytically, we can compute an oracle bound on performance, verify every diagnostic by hand, and attribute every observed difference between the two models to exactly one cause: the presence or absence of history in the input. In a realistic robotics experiment, dozens of confounds (network capacity, optimization noise, data volume, reward shaping) move simultaneously, and clean attribution becomes impossible. The pedagogical value of a system you can fully understand is that it lets you see the criteria working before you apply them to systems you cannot.

The environment's transition is a discretized double integrator with damping:

where:

• $v_t$: the cart's velocity at time $t$
• $p_t$: the cart's true position at time $t$
• $a_t \in [-1, 1]$: the acceleration command the agent chooses at time $t$
• $d_t$: a random disturbance force that keeps the control problem honest (without it, doing nothing would be optimal)
• $c$: the damping coefficient
• $\Delta t$: the timestep

The agent observes $o_t = p_t + \epsilon_t$, where $\epsilon_t$ is sensor noise.

Reading the equations physically: the first line says velocity updates under the commanded acceleration, a random disturbance, and a damping term proportional to current velocity, so the cart naturally slows unless pushed. The second line says position advances by the new velocity, a semi-implicit discretization choice that keeps the simulation stable at modest timesteps. Together they form a double integrator, the canonical minimal model of mechanical motion, dressed with just enough realism (damping, disturbance, noise) that none of our conclusions will feel like artifacts of a frictionless textbook fantasy.

We will train two models on identical data:

• Model M (memoryless): predicts the next position from the current position and action alone, $o_{t+1} \approx w^\top [1, o_t, a_t]$, where $w$ is the fitted weight vector.
• Model H (history-based): predicts from the last three positions and the action, $o_{t+1} \approx w^\top [1, o_t, o_{t-1}, o_{t-2}, a_t]$, where $w$ is the fitted weight vector.

Both are linear models fit by ridge regression, so the comparison changes one factor: how much observation history is available. Model M cannot explicitly reconstruct velocity. Model H has enough inputs to form a noisy velocity estimate, but whether three observations are sufficient is a result to test, not an assumption.

Why three positions specifically? Two consecutive noisy positions give a crude finite-difference velocity estimate, but the noise in each observation contaminates the difference heavily. Three positions begin to average that contamination down, giving the ridge fit enough signal to recover a usable implicit velocity feature while keeping the feature vector small enough to inspect by eye. The choice also foreshadows a general principle: the window length needed for sufficiency scales with how quickly the hidden state can be inferred from observations relative to the observation noise, a trade-off that filtering theory makes precise and that we revisit in Part II.

All modeling and numerical computation in this chapter uses NumPy; Matplotlib renders the figures. First, the environment and a data-collection routine.

The training data comes from uniformly random actions. This choice matters: random exploration spreads probability mass across the action space, which gives both models their best possible shot at action conditioning. Later, in the counterfactual experiment, we will see what happens when this generosity is withdrawn.

Note also what the random policy buys us beyond coverage: because actions are independent of states, the observational and interventional distributions coincide in this dataset, satisfying the exogeneity condition from Criterion 2 by construction. Any action-conditioning failure we observe later can therefore be attributed to the model rather than to confounding in the data. This is the experimental-design analogue of a randomized trial, and it is worth remembering that such cleanliness is a luxury purchased with exploration budget, one that real systems rarely enjoy.

Next we assemble regression datasets from the trajectories and fit both models by ridge regression. Each training example is a window of past observations plus the action taken, with the next observation as the target.

Before looking at any score, read the weights: they encode what each model believes about physics. The history model's coefficients on consecutive positions combine into an implicit velocity estimate, so its action coefficient should approach the true one-step displacement per unit action, roughly $\Delta t^2 = 0.01$. The memoryless model cannot represent velocity at all, so its action coefficient absorbs whatever the data's correlations allow and no more.

This weight-inspection habit generalizes far beyond linear models. Whenever your model class admits human-readable parameters, reading them before reading scores converts an opaque fitting exercise into a hypothesis test: you wrote down what correct physics would imply, and now you check whether the fit agrees. With neural networks the parameters stop being readable, but the practice survives in modified form as probing, which is exactly the diagnostic Criterion 1 recommended. The underlying discipline is the same: interrogate the representation directly instead of inferring its contents solely from end-task behavior.

We evaluate one-step prediction error on held-out random-action trajectories, then run the incremental-history diagnostic: does giving the memoryless model more history fix it?

Read @fig-history-sweep together with the printed numbers above. Error falls by about 20% from a one-observation window to a six-observation window, and it continues falling at the right edge. The test therefore rejects the convenient claim that three observations are already sufficient. It does show that the one-observation representation discards predictive information. The next experiment asks whether that modest one-step difference grows over long recursive rollouts.

One-step scores flatter both models. Now we let each model predict autoregressively: feed back its own prediction as the next input, for fifty steps, against held-out ground truth. This is the open-loop version of what a planner experiences.

Read the vertical axis: it is logarithmic. Both models deteriorate sharply, from errors near $0.002$ at one step to between $0.42$ and $0.44$ at step 50. The history model retains an advantage throughout, but the absolute gap does not explode; it becomes small relative to the shared long-horizon error. The defensible conclusion is not that one representation compounds while the other remains stable. It is that one-step validation understated the failure of both models at planning-length horizons.

This is also a useful negative result. Extra history improves the starting point, yet recursive prediction eventually overwhelms that advantage. State sufficiency and rollout stability are different properties: a better state can reduce error without making the learned transition stable under repeated self-input.

Now the decisive test. We place each model inside a model-predictive controller that plans by random shooting: sample many candidate action sequences, simulate each through the model, keep the sequence with lowest predicted cost, execute only its first action, observe, and replan. The task is regulation, holding the cart near the origin under random disturbance forces, with cost $p^2 + 0.05\,a^2$ per step.

Random shooting is the simplest possible planner, and its simplicity makes the experiment interpretable. It can still exploit model error, but it searches less aggressively than a gradient optimizer. Replanning provides error correction: executing one action and consulting fresh observations limits how long imagination errors persist. The experiment will tell us whether the representation difference matters after that feedback correction.

We compare four agents: a blind policy that always outputs zero force, the memoryless model driving the planner, the history model driving the planner, and an oracle planner that cheats by simulating the true plant (including the hidden velocity) during its search. The oracle marks the ceiling achievable with perfect state knowledge; the gap between it and the learned models is the price of partial observability.

The baseline ladder matters as much as the headline numbers. The blind policy establishes what doing nothing achieves, so any claimed benefit of modeling must clear that bar. The oracle establishes what perfect information would achieve, so the distance from oracle to history model quantifies the remaining headroom attributable to estimation error rather than to fundamental difficulty. Without both anchors, a cost figure in isolation supports no conclusion: a mediocre result might reflect a hard task or a weak model, and only the anchors distinguish the two.

Notice what this experiment demonstrates about the relationship between the criteria. The offline tests found a representation weakness, but closed-loop replanning largely compensated for it on this task. Had we run only the one-step diagnostics, we might have predicted a much larger control deficit. Had we run only the closed-loop comparison, we would miss the reason the oracle still has an advantage and the possibility that a less forgiving task would expose it. The battery earns its length by revealing both the deficiency and the task's tolerance for it.

Finally, we withdraw the generous training data. Suppose the agent's predecessor operated a cautious feedback policy, $a_t = \mathrm{clip}(-2\,p_t + \eta_t, -1, 1)$ with small Gaussian noise $\eta_t$, so the logged data contains only gentle corrections around the origin. We retrain both models on this narrow diet and then interrogate them with two test sets: held-out transitions from the same cautious policy, and transitions under the wide random actions we used before.

This scenario mirrors the most common real-world data situation. Deployed systems log their own behavior, and deployed behavior is produced by controllers, which are smooth functions of state. The result is a dataset in which actions are nearly deterministic responses to observations, spanning only a thin manifold of the state-action space. Everything the qualification framework warns about concentrates in this setting: actions are confounded with states (breaking exogeneity), the action-space coverage collapses (limiting counterfactual validity), and the resulting models can look superb on held-out logs while being worthless for planning. The experiment below quantifies exactly how large the gap can be.

@fig-action-margin-probe shows the interventional test in its simplest form. The wide-data coefficient is about $+0.011$, close to the expected $+0.01$. The narrow-policy coefficient is about $-0.078$: it has the wrong sign and roughly seven times the magnitude. Aggregate error shows that the model is fragile; this probe identifies the broken mechanism. The narrow fit has mistaken a policy correlation, stronger corrective actions occurring in states already moving toward recovery, for the physical effect of the action itself.

The key parameters for the world-model qualification experiments are:

• DT: Timestep of the discretized plant. It sets the scale of one-step displacement and appears in both the true dynamics and the fitted models.
• DAMPING: Damping coefficient $c$ of the cart. Stronger damping makes velocity decay faster, shrinking the benefit of a history-based state.
• OBS_NOISE: Standard deviation of sensor noise. Together with fresh process disturbance, it contributes to a lower-bound reference for one-step prediction error.
• DIST_STD: Standard deviation of the random disturbance force. It keeps the regulation task non-trivial; without it, zero control would be optimal.
• lam: Ridge regularization strength in fit_ridge. It stabilizes the closed-form solve when features are highly correlated.
• HORIZON: Number of steps in the recursive rollouts used to measure compounding error.
• PLAN_HORIZON: Lookahead horizon of the random-shooting MPC planner.
• N_CANDIDATES: Number of candidate action sequences sampled at each replanning step.
• EPISODE_STEPS and N_EPISODES: Length and count of the closed-loop evaluation episodes, which together set the noise level of the cost comparison.

Several of these parameters repay deliberate manipulation once you have the notebook running, because each one tunes a different lesson. Raising DAMPING toward large values makes velocity die quickly, so recent position alone approximates the full state and the memoryless model's handicap shrinks; lowering it does the opposite and widens the gap. Increasing OBS_NOISE raises the floor beneath both models and makes the history window's averaging role more valuable. Reducing N_CANDIDATES or PLAN_HORIZON degrades the planner uniformly, illustrating that planner quality and model quality are separable axes of the closed-loop result. Experiments like these convert the parameter list from documentation into a small laboratory.

Stepping back, the experiment compressed the qualification test into a few minutes of CPU time, and its untidy results are the point:

• State sufficiency was probed cheaply with a history sweep. Extra history helped, but the curve did not flatten, so none of the tested windows earned a sufficiency claim.
• Action conditioning was identified under wide random actions and failed under narrow policy data: the learned action gain reversed sign despite low in-distribution error.
• Counterfactual validity was measured as error inflation off the training distribution, and it reframed "model quality" as a property of model-plus-data coverage.
• Decision usefulness required the environment in the loop. Both learned controllers beat the blind baseline, history helped modestly, and neither matched the oracle.

Notice the division of labor across the battery. Three of the four tests ran entirely offline, on recorded data, at negligible cost, and they localized the failure precisely: to the representation, to the data coverage, or to neither. Only the fourth required interacting with the environment, which in real systems means spending hardware time, risking wear, and accepting safety exposure. This economics is the quiet argument for the battery's ordering: exhaust the cheap tests first, and arrive at the expensive one with a hypothesis rather than a shrug. Closed-loop evaluation is irreplaceable, but it is also the scarcest resource in most applied programs, and the earlier criteria exist partly to protect it.

Two habits from this chapter are worth carrying forward permanently. First, always report the noise floor or an irreducible-error baseline alongside model error, so that "good" has a reference point. Second, never let a model be graded solely by the distribution its data came from, because a planner's job is to leave that distribution.

The qualification test earns its keep by catching failures that loss curves miss, but it is worth stating plainly what it does not settle. The four criteria are necessary, not sufficient. A model can pass every test we ran and still fail in deployment: our experiment used a linear plant, dense random exploration, a known reward, and a planner matched to the model's interface. Real systems violate all four comforts at once. Partial observability can be structural rather than incidental, with hidden variables that no finite history window reconstructs. Exploration budget is scarce, so the wide-coverage data that secures action conditioning may be expensive or dangerous to gather. And the planner itself introduces a moving target: improve the optimizer and you change which regions of the model get stressed, potentially invalidating yesterday's qualification.

Each of these violated comforts is worth naming concretely, because together they sketch the gap between this chapter's laboratory and industrial practice. Structural partial observability means the belief-state remedy may require a posterior over high-dimensional or poorly modeled hidden variables, where our cart needed only a scalar velocity. Scarce exploration means the random-action luxury must be replaced by carefully engineered excitation signals, safe exploration algorithms, or transfer from simulation, each of which reintroduces the confounding we sidestepped. And the moving-target problem means qualification is a recurring activity tied to the deployment configuration, not a certificate issued once at training time. Organizations that treat model evaluation as a launch gate, rather than a continuous process coupled to the planner and the data pipeline, rediscover this lesson the expensive way.

There is also a deeper conceptual limit worth naming. Decision usefulness is relative to a task, so "qualified" is never a global verdict. A model that regulates a cart to the origin may be useless for tracking a moving target, and a model superb at half-second video prediction may be unusable for minute-scale planning. Practitioners therefore qualify models per deployment envelope: a declared region of states, actions, horizons, and tasks within which the four criteria have been verified. Outside the envelope, all bets are off, and a central engineering discipline of Part XII is building agents that know where the envelope ends.

The envelope concept deserves a final elaboration because it reframes what a "qualified model" document should contain. A serious qualification record specifies not just scores but scope: the state regions probed, the action ranges tested, the horizons rolled out, the reward structures evaluated, and the baselines compared against. Two teams declaring "our model passed qualification" may have verified disjoint things, and only the envelope descriptions reveal it. This is the same discipline that structural engineering applies to load ratings: a bridge is not "strong," it is strong for stated loads, and the sign stating the limit is part of the safety system. Agents that monitor their own distance from the qualified envelope, and degrade gracefully or defer to humans when approaching its edge, are the subject of Part XII.

These limits explain why the qualification test is a recurring motif rather than a one-time gate. Every major lineage in this book, from system identification through model-based RL to foundation-scale video models, rediscovers the same four questions in its own dialect, and each generation of architectures shifts the frontier of which failures are easy to catch. What the test provides is a shared language for those conversations: when someone reports a new world model, the productive questions are always the same four. What is in the state? Do actions cause anything? Where do the predictions stop being trustworthy? And does the model improve behavior?

• A world model is qualified not by prediction loss alone but by four criteria: state sufficiency, action conditioning, counterfactual validity, and decision usefulness.
• State sufficiency is a property of the representation; its signature is irreducible error that added history cures. Belief states and recurrent latents are the standard remedies.
• Action conditioning must be verified interventionally, because logged data entangles actions with the behavior policy that produced them.
• Counterfactual validity is relative to a declared region of state-action space. In this experiment, changing the behavior policy inflated off-policy error by roughly two orders of magnitude while in-policy scores remained small.
• Closed-loop evidence is the final decision test: compounding drift, model exploitation, and policy-induced distribution shift can be invisible to one-step validation and consequential inside an agent.
• In the worked experiment, extra history improved prediction and control only modestly, while feedback let both learned controllers beat the blind baseline. The qualification battery rejected stronger conclusions that the original hypothesis invited.

The natural next question is what kinds of world models exist and where each sits along the axes this chapter exposed: observable versus latent states, fixed versus learned dynamics, prediction-centric versus decision-centric objectives. The next chapter, Taxonomies of World Models, takes up that classification.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about the world-model qualification test and the four criteria that separate true world models from mere forecasters.