The World-Model Qualification Test › Criterion 1: State Sufficiency

A world model predicts the future from an internal state. The first qualification question is whether that state contains enough information. Formally, we say a state representation is sufficient when the future is conditionally independent of the past given the state and the planned action:

where:

• $o_{t+1:T}$: the sequence of future observations from time $t+1$ up to horizon $T$
• $r_{t+1:T}$: the sequence of future rewards over the same horizon
• $s_t$: the model's internal state at time $t$
• $a_{t:T}$: the planned actions from time $t$ up to the horizon
• $o_{\le t}$: the full history of observations up to and including time $t$
• $a_{<t}$: the historical actions taken before time $t$

That is, once you know $s_t$ and the future actions, the raw history $o_{\le t}$ adds nothing. This is the predictive core of the Markov assumption, and it is a property of the representation rather than of the environment alone: a fully observable environment can still be modeled inadequately if the model compresses away a relevant variable.

The last clause is the one students most often miss, so let us dwell on it. Markovianity is usually introduced as a property of environments: either the physical system's current state determines its future or it does not. But the qualification question here is about the model's internal state, which is a lossy summary chosen by whoever designed or trained the model. Two models of the same perfectly Markovian cart can differ sharply in sufficiency if one keeps velocity information and the other discards it. Sufficiency therefore depends jointly on the environment and the representation: the environment determines what information exists, and the representation determines how much of it survives compression. When we say a model "has" a sufficient state, we mean the compression preserved everything the future depends on, which is an empirical claim about the trained artifact, not a design intention.

Why does this deserve its own criterion rather than being folded into "accuracy"? Because insufficiency produces a characteristic signature that accuracy numbers obscure: the model's residual error is irreducible from the outside. If a model of a car tracks only position and not velocity, no amount of training improves its prediction of where the car will be in one second, because the missing variable matters. The error looks like noise. It is structure the representation discarded.

Confusing discarded structure with irreducible noise is an expensive mistake in applied world modeling. Sensor noise is unpredictable given the available inputs. Insufficient-state error can look similar from the outside: residual variance plateaus, the loss curve flattens, and ordinary training signals say "training complete." Changing the input representation can reduce insufficient-state error but cannot remove sensor noise. Varying the input history is therefore a useful diagnostic: a lower plateau shows that the shorter representation discarded predictive structure. An unchanged plateau is weaker evidence. Optimization failure, limited model capacity, and finite data must also be ruled out before calling the remainder a noise floor.

Two common remedies anchor large portions of this book.

• Carry more history. Feed the model a window of past observations, or a recurrent summary of all past observations, so that latent quantities such as velocities can be inferred. Recurrent state-space models do exactly this, as we will see in Part V: World-Model Architectures.
• Learn a belief state. Maintain an explicit posterior over the hidden physical state given the observation history. This is the classical filtering approach, covered in detail in the Bayesian filtering chapter of Part II: Inference, Decisions, and Control.

These remedies embody different choices about where inference should happen. Carrying history delegates the work to the transition model: the network must extract velocity-like features from raw observation sequences. A belief state makes inference explicit: a filtering step maintains a posterior over hidden variables, and the dynamics operate on that posterior. History-based models often trade interpretability for flexibility; explicit filters trade stronger modeling assumptions for structured uncertainty. State observers and Kalman-style filters became central tools in modern control engineering, while many deep world models favor learned recurrent summaries. Part II and Part V develop both lineages in depth.

How do you test for sufficiency in practice? The cheapest diagnostic is incremental: train the same architecture while conditioning on progressively more history, and watch whether one-step error keeps falling. If adding the last three observations sharply reduces error but adding the last ten changes nothing, the model has extracted what history can offer. If error falls monotonically forever, the model is still starving. A sharper diagnostic is probing: train a small auxiliary regressor to predict known-relevant hidden quantities (velocity, contact state, the position of an occluded object) from the model's internal state. If the probe fails, the information is gone and no downstream improvement will recover it. We will develop these diagnostics properly in Part XI: Evaluation and Understanding.

Both diagnostics reward a little interpretation. The incremental-history diagnostic is essentially a nonparametric estimate of how much predictive information remains in the observation history beyond what the model already uses, so a flat curve is evidence that the architecture's input window is not the bottleneck. The probe diagnostic is stronger but requires domain knowledge: you must know which hidden quantities matter before you can check whether the state encodes them. In practice the two compose well. Use the history sweep to establish that more input would help or not, then use targeted probes to identify which latent variables are missing, since knowing that something is missing is less actionable than knowing what it is. A probe that recovers velocity from a recurrent state but not contact state tells you precisely which physical phenomenon your representation has failed to capture, and that diagnosis points directly at the remedy.