What Is a World Model? › The Anatomy of a World Model

Having defined the role, we can now dissect the organ. Nearly every world model, from a Kalman filter's internal equations to a billion-parameter latent video predictor, decomposes into the same four functional components. Keeping them conceptually distinct is the single most valuable habit you can form in this field, because a remarkable fraction of published confusion comes from blurring them together. When a paper claims its model "learns better state representations," ask which component improved: the encoder mapping observations inward, the transition mapping states forward, or the interface between them. When a system fails, ask which component failed. Diagnosis begins with anatomy.

The key components of a world model are:

• Observations: what the agent actually receives from sensors, typically partial, noisy, and delayed.
• State: the information sufficient to summarize the past for purposes of predicting the future.
• Dynamics: the rules by which state evolves, optionally as a function of actions.
• Agent structure: the model's account of rewards, of the agent itself, and of other agents sharing the world.

We examine each in turn, then see how they assemble into a formal object.

An observation $o_t$ is whatever crosses the sensory boundary at time $t$: camera pixels, joint angles, lidar returns, the price ticker, the words in a conversation. Observations have three universal vices.

They are partial. A camera sees surfaces, not the objects behind them; a thermometer reads one point of a room. Partiality is not an engineering defect to be engineered away; it is a consequence of living inside the world you observe. No sensor array mounted on an agent can see through every occluder, read every mind, or measure every unobservable quantity like an object's mass or another driver's intention. They are noisy. Sensors corrupt signals with measurement noise, and the corruption is often state-dependent: cameras saturate in bright light, encoders slip under load, financial feeds drop ticks during volatility. Noise that depends on state is especially treacherous because it masquerades as dynamics. They are delayed and asynchronous. Real systems deliver stale information, sometimes from different clocks. A robot's camera frame may describe the world forty milliseconds ago, by which time the arm has already moved. Delays turn even perfect sensors into sources of uncertainty about the present.

Because of these vices, the observation stream $o_1, o_2, \dots$ is generally not a sufficient summary of history. Two different worlds can produce identical observation sequences yet demand different responses. Any serious world model therefore maintains something richer than the latest observation, which brings us to state. It is worth being explicit about the direction of the fix: since observations cannot be made sufficient, the agent must construct sufficiency internally, by remembering, filtering, or inferring. Everything between the raw sensor feed and the predictive core of a world model exists to perform this construction.

The Markov property is the hinge on which the whole handbook turns, so let us make it concrete with the running example. Our pendulum's physics is fully determined by two numbers: the angle $\theta_t$ and the angular velocity $\omega_t$. If you know both, the entire prior trajectory is irrelevant to predicting the future. The pair $(\theta_t, \omega_t)$ is a Markov state. Notice what Markovianity buys: it collapses an unbounded past into a fixed-size summary, which means prediction becomes a function of the present alone rather than of ever-growing histories. Without it, every predictor must carry the whole past with it, and computation grows without bound as time proceeds.

But our frosted glass only lets you measure a blurred, noisy version of the angle. The angle alone is not Markov: a pendulum momentarily at 30 degrees might be swinging upward or downward, and the two cases evolve completely differently. The missing information lives in the history. To predict well from observations alone, you must reconstruct or remember something like velocity, either by keeping a memory of recent observations or by maintaining an explicit internal estimate. These are the two classical strategies, memory-based and estimate-based, and nearly every architecture in this book is a variation on one of them: sequence models remember, filters infer, and modern recurrent world models do a learned blend of both.

This is exactly what classical engineering does with the Kalman filter and what modern deep agents do with recurrent latent states, and Part II of this handbook develops both threads rigorously: Markov and partially observable decision processes in its opening chapter, Bayesian filtering and belief states in the second. For now, hold onto the key structural insight:

A world model's internal state need not match the environment's physical state. It only needs to be sufficient: to support the predictions the task requires.

This license is enormously powerful. The latent state of a learned world model may be a 256-dimensional vector with no human-interpretable coordinates, yet if it supports accurate action-conditioned prediction, it functions as a state. Conversely, a physically exact state that omits one task-relevant variable (say, the position of a hidden obstacle) fails as a world model for that task. Sufficiency, not fidelity, is the standard. One subtlety deserves attention: sufficiency is defined relative to a task and its horizon. A state sufficient for balancing a pendulum for ten steps may be insufficient for planning a swing-up over five hundred steps, because longer horizons expose more of the future to whatever the state forgot. When you change the task, audit the state.

The dynamics component encodes the rule of motion. In discrete time, a deterministic model writes

$$s_{t+1} = f(s_t, a_t),$$

while a stochastic model specifies a conditional distribution,

$$s_{t+1} \sim p(\cdot \mid s_t, a_t).$$

Several distinctions hide inside these compact forms, and each carries modeling weight.

Deterministic versus stochastic. Real environments contain genuine randomness (a flipped coin, an opponent's choice) as well as apparent randomness from unmodeled variables. A stochastic model represents this with an explicit noise distribution; a deterministic model must either average over randomness, losing the ability to express uncertainty, or fail outright on multimodal futures. Choosing between them is a modeling decision with downstream consequences for planning, because planners that assume determinism can be badly fooled by stochastic worlds. Consider a deterministic planner facing a stochastic environment: it computes one optimal trajectory, executes it, and treats every deviation as noise to be corrected. Now consider what happens when the environment has two distinct modes, such as a door that may or may not be locked. A deterministic model averages the two futures into one impossible hybrid, and plans against the hybrid. Only a stochastic model can represent the fork, and only a stochastic planner can hedge across it.

Stationarity. We usually assume the dynamics rule does not change over time. This is an assumption, not a law. Environments drift, agents wear out, opponents adapt. Non-stationarity breaks models trained on old data, and handling it is a live research problem rather than a settled technique. Even the appearance of stationarity can deceive: a process may look stationary over the window you observed and drift outside it, which is why validation windows matter as much as training windows.

Continuous versus discrete time. Physics runs continuously; computers step discretely. Classical control theory works in continuous time with differential equations, while learned world models almost always operate at a fixed discretization $\Delta t$. The choice of $\Delta t$ quietly shapes everything downstream: too coarse and fast phenomena alias away, too fine and long-horizon rollouts become computationally expensive and accumulate error over more steps. The two goals pull against each other, because compounding error grows roughly with the number of steps rolled out, so finer discretization buys physical fidelity at the price of more opportunities for error to compound. Some modern approaches sidestep the dilemma with multi-step or adaptive-time predictions, but the basic trade-off governs most practical choices.

Known form versus learned. Sometimes dynamics come from physics: Newton's laws give us the pendulum equation $\ddot{\theta} = -(g/l)\sin\theta - (c/ml^2)\dot\theta + u/(ml^2)$ up to a few constants. Often they must be learned from data, which is the province of system identification for parametric models and of representation learning for latent ones. Part II, Chapter 3 covers system identification, and Part V covers neural architectures for learned dynamics. Between these poles lies a productive middle ground: grey-box modeling, in which physics fixes the functional form and data estimates the constants, or physics-informed networks in which known laws regularize learned components. Most successful industrial models live in this middle ground rather than at either pure extreme.

A bare dynamics model predicts motion. Agents need one more ingredient to convert prediction into preference: a model of reward, $r(s_t, a_t)$ or $r(o_t)$, scoring how good each situation or action is. With dynamics and reward together, the agent can evaluate hypothetical futures: simulate a course of action, sum the rewards along it, and compare courses. This pairing is precisely what model-based reinforcement learning exploits, and it is why reward modeling deserves first-class status rather than being treated as an afterthought bolted onto a dynamics model. The pairing also introduces a new failure surface: errors in the reward model corrupt decisions even when the dynamics are perfect, and vice versa. A planner is only as honest as the least reliable component it trusts, and disentangling which component lied after a bad outcome is a diagnostic skill worth cultivating early.

The fourth component is subtler: the model's account of agents, including the agent itself.

• Self-model: a car's world model includes its own braking capability, turning radius, and sensor blind spots. An agent that cannot predict its own body's response to commands cannot plan with its body. Robotics calls parts of this the body schema; control theory calls it the actuator model. The self-model is easy to overlook because it feels like part of the agent rather than part of the world, but from the perspective of planning, your own actuation limits are features of the environment you must predict: they determine which imagined action sequences are physically executable.
• Other-agent models: in traffic, games, and conversation, the environment contains entities that react to you. Predicting them requires modeling their goals, beliefs, and likely responses, a capacity cognitive science associates with theory of mind. Multi-agent world models remain one of the hardest open areas, and we revisit them in Part IV and Part X. The difficulty is structural, not incidental: once others react to your predictions, the fixed-target assumption underlying ordinary supervised learning dissolves, and the problem acquires a game-theoretic character.

Notice the recursion this creates: the agent's actions change the world, the changed world changes the agent's observations, and the observations update the model. A world model embedded in a closed loop is modeling a system that contains itself. This reflexivity is harmless for the pendulum but consequential for economics, ecology, and any setting where predictions influence the thing predicted. A famous illustration is traffic navigation apps: when enough drivers believe the same predicted fast route, the route stops being fast. The model's output becomes part of the system's input, and stable, accurate prediction may cease to exist at all.

The table below summarizes the anatomy and the question each component answers.

Assembling the pieces, the canonical formal object underneath most of this book is the partially observable decision process: states evolving by $p(s_{t+1} \mid s_t, a_t)$, observations emitted by $p(o_t \mid s_t)$, rewards $r$, all estimated by an agent whose internal model mirrors this structure with hats: $\hat{p}$, $\hat{r}$, and an internal state $\hat{s}_t$ updated from the observation stream. Part II, Chapter 1 gives this object its full formal treatment. Here it suffices to see that "world model" names the agent-side mirror of this structure, however crude the mirror may be. The mirroring metaphor is worth keeping: the agent never observes the true process directly, only its reflections, and the art of world modeling is building a mirror accurate enough, in the right places, to steer by.