What Is a World Model?

You are driving toward a bridge on a cold morning. The surface looks wet, but it might be ice. Before you reach it, something happens inside your head: you imagine pressing the brake, feel the imagined slide of the car, estimate how long it would take to stop, and ease off the accelerator instead. You never touched the brake. You ran the experiment somewhere else, in a model of the world carried inside you, and you acted on what that model told you.

Notice how much happened in that brief moment of rehearsal. You consulted stored knowledge about friction and temperature. You generated a hypothetical future rather than waiting for the real one. You compared at least two candidate actions, braking and coasting, and assigned them different imagined outcomes. And crucially, you trusted those imagined outcomes enough to let them override the direct evidence of your eyes, which said only "the road looks wet." That last step is the one that separates a mere observer from an agent: the willingness to act on prediction rather than perception alone.

That inner rehearsal is the core idea behind this entire handbook. A world model is a mechanism, biological or artificial, that predicts how situations evolve in response to actions, and does so well enough to improve the decisions made with it. In Chapter 1, Building a First World Model, you built a small working instance of this idea and watched an agent plan against its own learned dynamics. This chapter steps back and asks the definitional question squarely: what exactly counts as a world model, what parts does one have, and where does it begin and end relative to its neighbors, like generative models and simulators?

These are not idle taxonomy questions. When researchers argue about whether a video generator "understands physics," or whether a reinforcement learning agent has "learned the environment," they are really arguing about definitions. Getting precise now will save you from confusion for the rest of the book, because nearly every later chapter refines, extends, or stress-tests one of the concepts introduced here. Precision also has practical stakes: if you cannot say what a world model is, you cannot evaluate one, compare two candidates, or diagnose why yours fails. Definitions are the instruments by which the rest of this handbook measures everything else.

We will use one running example throughout: a pendulum viewed through frosted glass. It is small enough to hold in your head, yet it contains every major ingredient: hidden state, noisy observation, learned dynamics, prediction, simulation, and control. By the end of the chapter you will have built and stress-tested a complete miniature world model of it. The pendulum is deliberately humble. Its simplicity is the point: when every ingredient is visible at once, you learn to see the ingredients, and later, when they hide inside billion-parameter networks, you will know what to look for.

There are two complementary answers to "what is a world model?", and mature practitioners hold both at once. The first is an engineering specification: a functional contract the model must satisfy. The second is a scientific and philosophical claim about what it means for a mind or a machine to carry a model of the world at all. The two traditions ask different questions, use different vocabularies, and occasionally disagree about borderline cases, which is precisely why holding both is valuable. An engineer who ignores the philosophical tradition builds systems that predict without representing; a philosopher who ignores the operational test praises systems that represent without predicting anything useful.

The operational definition treats "world model" as a role, not an object. Anything that plays the role counts, regardless of its internal machinery. This is the same move biology makes when it defines a wing by what it does for flight rather than by what it is made of: feathers, membranes, and engineered airfoils are all wings because they all generate lift. By analogy, a lookup table, a differential equation, and a recurrent neural network are all world models if they all support action-conditioned prediction good enough for control.

Every word in that definition is doing work. Let us unpack the three clauses that matter most.

It predicts consequences of actions. A passive forecaster of tomorrow's weather is a predictor, but a world model answers counterfactual, action-conditioned questions: what would happen if I did this? Formally, if the environment is a system with state $s_t$, actions $a_t$, and transitions governed by some true dynamics $p(s_{t+1} \mid s_t, a_t)$, the agent's world model maintains an approximation $\hat{p}_\theta(s_{t+1} \mid s_t, a_t)$, together with auxiliary pieces we will catalog shortly. The conditioning on $a_t$ is what separates a world model from a mere video forecaster, and we will return to this boundary later in the chapter. It is worth pausing on how demanding this clause really is. A passive forecaster only needs to be right about the futures that actually happen, weighted by how often they happen. An action-conditioned model must be right about futures that never happen, including futures under actions nobody has ever taken, because planning requires evaluating actions before committing to them. The model must extrapolate across the space of actions, not merely interpolate across the stream of time.

"Accurately enough" is relative to decisions. Perfection is neither achievable nor required. A chess player's mental model of the board is wildly incomplete as physics, yet superb for choosing moves. An error that shifts a long-horizon forecast by a millimeter may be irrelevant; a small bias in braking-distance estimation may be fatal. This is why the operational definition refuses to specify a universal accuracy threshold. The right threshold is set by the decisions downstream, a principle we will formalize as decision sufficiency below. Think of accuracy as a budget to be spent where it matters. A warehouse robot's world model can be sloppy about the color of boxes and sloppy about positions beyond its reach, but it had better be precise about gripper trajectories near shelving. Every modeling choice allocates finite data, capacity, and attention somewhere, and the operational definition says: allocate them toward the decisions.

It improves decisions made with it. This clause excludes systems that predict beautifully but mislead their planners. A model can have low prediction error on the data distribution and still cause catastrophic failure when an optimizer searches for actions that exploit its blind spots. The gap between predicting well and deciding well is one of the central tensions of model-based reinforcement learning, and it will recur throughout Part VI and Part VII of this handbook. If you remember only one sentence from this chapter, make it this one: the value of a world model is realized only through the closed loop, and closed-loop value is a different quantity from open-loop accuracy.

Under this definition, the term deliberately spans an enormous range: a one-step linear regression used by a controller, a recurrent latent dynamics network imagined forward by Dreamer-style agents, a monkey's grasp of where a thrown ball will land, and a full physics engine. They differ wildly in fidelity, scope, and mechanism, but they occupy the same functional role. This breadth is a strength, not a sloppiness. It lets us transfer insights across scales: lessons learned from fitting a linear model to fifty seconds of pendulum data reappear, transformed but recognizable, in the training of planetary-scale video predictors.

The second tradition asks a deeper question: what does it mean for an organism to have a model of the world? This lineage is older than machine learning. Kenneth Craik argued in The Nature of Explanation (1943) that thought works by carrying a "small-scale model" of external reality inside the head, allowing an organism to try out alternatives, anticipate outcomes, and react to future situations before they arrive. Craik's phrase deserves attention: he described the mental model as small-scale, meaning not a copy of the world but a compressed, manipulable surrogate, cheap enough to run many experiments in quickly. That compression-versus-fidelity trade-off is exactly the trade-off you will face when choosing the dimensionality of a learned latent state. Edward Tolman's experiments on rats (1948) suggested that animals navigating mazes acquire cognitive maps, structured representations of spatial layout that go beyond stimulus-response habits. Tolman's rats took shortcuts to rewards they had never traveled to directly, behavior that could not be explained by memorized stimulus-response chains and demanded something map-like inside the animal. Later theorists such as Johnson-Laird developed these ideas into the "mental models" framework in cognitive psychology, studying how people reason about physical systems, deductive problems, and other minds by constructing and inspecting internal surrogates.

On this view, a world model is characterized less by what it does for a controller and more by what it represents: entities, relations, causal structure, and the way they unfold in time. This raises questions the operational definition happily ignores:

• Representation: Is the model a literal analog of the world's structure, or an arbitrary internal code that merely behaves as if it were? A map drawn to scale and a set of driving directions encode the same city differently, and both work. Whether internal codes must resemble the world, or merely serve it, remains contested.
• Grounding: How do the model's symbols connect to sensory experience and physical reality? A symbol that never touches observation is speculation; grounding is the discipline that ties representation to evidence.
• Scope: Does the model cover the observer's own body and mind, other agents, or only the inert environment? Human models conspicuously include the self and other people, and much of human competence depends on that inclusion.
• Acquisition: Is the model innate, learned from experience, or some interplay of both? Infants appear to arrive with priors about objects and solidity, then refine them dramatically through interaction, suggesting the answer is rarely either extreme.

You do not need to resolve these questions to build useful systems, and this handbook is primarily operational. But the philosophical tradition matters for two practical reasons. First, it explains why the field expects certain properties, like compositional generalization and causal robustness, from advanced world models: those are properties of human mental models that purely correlational predictors lack. When we demand that a model handle novel combinations of familiar elements, we are importing an expectation born from watching minds do it. Second, it supplies the vocabulary in which much of the research frontier is argued, especially in cognitive science collaborations and discussions of foundation models in Part IX: Foundation and World-Action Models. Reading that literature without the Craik-Tolman lineage is like reading physics without knowing what "energy" meant before thermodynamics formalized it.

Throughout this book we will move between the two registers. When we say a model "is a world model," we usually mean the operational test. When we ask whether a model "understands" its world, we are invoking the philosophical one. Keeping the registers distinct prevents a common species of pointless argument, in which one party applies the functional test and the other the representational one, and both leave convinced the other missed the point. The next chapter, The World-Model Qualification Test, turns the operational definition into concrete criteria a candidate model must pass.

Having defined the role, we can now dissect the organ. Nearly every world model, from a Kalman filter's internal equations to a billion-parameter latent video predictor, decomposes into the same four functional components. Keeping them conceptually distinct is the single most valuable habit you can form in this field, because a remarkable fraction of published confusion comes from blurring them together. When a paper claims its model "learns better state representations," ask which component improved: the encoder mapping observations inward, the transition mapping states forward, or the interface between them. When a system fails, ask which component failed. Diagnosis begins with anatomy.

The key components of a world model are:

• Observations: what the agent actually receives from sensors, typically partial, noisy, and delayed.
• State: the information sufficient to summarize the past for purposes of predicting the future.
• Dynamics: the rules by which state evolves, optionally as a function of actions.
• Agent structure: the model's account of rewards, of the agent itself, and of other agents sharing the world.

We examine each in turn, then see how they assemble into a formal object.

An observation $o_t$ is whatever crosses the sensory boundary at time $t$: camera pixels, joint angles, lidar returns, the price ticker, the words in a conversation. Observations have three universal vices.

They are partial. A camera sees surfaces, not the objects behind them; a thermometer reads one point of a room. Partiality is not an engineering defect to be engineered away; it is a consequence of living inside the world you observe. No sensor array mounted on an agent can see through every occluder, read every mind, or measure every unobservable quantity like an object's mass or another driver's intention. They are noisy. Sensors corrupt signals with measurement noise, and the corruption is often state-dependent: cameras saturate in bright light, encoders slip under load, financial feeds drop ticks during volatility. Noise that depends on state is especially treacherous because it masquerades as dynamics. They are delayed and asynchronous. Real systems deliver stale information, sometimes from different clocks. A robot's camera frame may describe the world forty milliseconds ago, by which time the arm has already moved. Delays turn even perfect sensors into sources of uncertainty about the present.

Because of these vices, the observation stream $o_1, o_2, \dots$ is generally not a sufficient summary of history. Two different worlds can produce identical observation sequences yet demand different responses. Any serious world model therefore maintains something richer than the latest observation, which brings us to state. It is worth being explicit about the direction of the fix: since observations cannot be made sufficient, the agent must construct sufficiency internally, by remembering, filtering, or inferring. Everything between the raw sensor feed and the predictive core of a world model exists to perform this construction.

The Markov property is the hinge on which the whole handbook turns, so let us make it concrete with the running example. Our pendulum's physics is fully determined by two numbers: the angle $\theta_t$ and the angular velocity $\omega_t$. If you know both, the entire prior trajectory is irrelevant to predicting the future. The pair $(\theta_t, \omega_t)$ is a Markov state. Notice what Markovianity buys: it collapses an unbounded past into a fixed-size summary, which means prediction becomes a function of the present alone rather than of ever-growing histories. Without it, every predictor must carry the whole past with it, and computation grows without bound as time proceeds.

But our frosted glass only lets you measure a blurred, noisy version of the angle. The angle alone is not Markov: a pendulum momentarily at 30 degrees might be swinging upward or downward, and the two cases evolve completely differently. The missing information lives in the history. To predict well from observations alone, you must reconstruct or remember something like velocity, either by keeping a memory of recent observations or by maintaining an explicit internal estimate. These are the two classical strategies, memory-based and estimate-based, and nearly every architecture in this book is a variation on one of them: sequence models remember, filters infer, and modern recurrent world models do a learned blend of both.

This is exactly what classical engineering does with the Kalman filter and what modern deep agents do with recurrent latent states, and Part II of this handbook develops both threads rigorously: Markov and partially observable decision processes in its opening chapter, Bayesian filtering and belief states in the second. For now, hold onto the key structural insight:

A world model's internal state need not match the environment's physical state. It only needs to be sufficient: to support the predictions the task requires.

This license is enormously powerful. The latent state of a learned world model may be a 256-dimensional vector with no human-interpretable coordinates, yet if it supports accurate action-conditioned prediction, it functions as a state. Conversely, a physically exact state that omits one task-relevant variable (say, the position of a hidden obstacle) fails as a world model for that task. Sufficiency, not fidelity, is the standard. One subtlety deserves attention: sufficiency is defined relative to a task and its horizon. A state sufficient for balancing a pendulum for ten steps may be insufficient for planning a swing-up over five hundred steps, because longer horizons expose more of the future to whatever the state forgot. When you change the task, audit the state.

The dynamics component encodes the rule of motion. In discrete time, a deterministic model writes

while a stochastic model specifies a conditional distribution,

Several distinctions hide inside these compact forms, and each carries modeling weight.

Deterministic versus stochastic. Real environments contain genuine randomness (a flipped coin, an opponent's choice) as well as apparent randomness from unmodeled variables. A stochastic model represents this with an explicit noise distribution; a deterministic model must either average over randomness, losing the ability to express uncertainty, or fail outright on multimodal futures. Choosing between them is a modeling decision with downstream consequences for planning, because planners that assume determinism can be badly fooled by stochastic worlds. Consider a deterministic planner facing a stochastic environment: it computes one optimal trajectory, executes it, and treats every deviation as noise to be corrected. Now consider what happens when the environment has two distinct modes, such as a door that may or may not be locked. A deterministic model averages the two futures into one impossible hybrid, and plans against the hybrid. Only a stochastic model can represent the fork, and only a stochastic planner can hedge across it.

Stationarity. We usually assume the dynamics rule does not change over time. This is an assumption, not a law. Environments drift, agents wear out, opponents adapt. Non-stationarity breaks models trained on old data, and handling it is a live research problem rather than a settled technique. Even the appearance of stationarity can deceive: a process may look stationary over the window you observed and drift outside it, which is why validation windows matter as much as training windows.

Continuous versus discrete time. Physics runs continuously; computers step discretely. Classical control theory works in continuous time with differential equations, while learned world models almost always operate at a fixed discretization $\Delta t$. The choice of $\Delta t$ quietly shapes everything downstream: too coarse and fast phenomena alias away, too fine and long-horizon rollouts become computationally expensive and accumulate error over more steps. The two goals pull against each other, because compounding error grows roughly with the number of steps rolled out, so finer discretization buys physical fidelity at the price of more opportunities for error to compound. Some modern approaches sidestep the dilemma with multi-step or adaptive-time predictions, but the basic trade-off governs most practical choices.

Known form versus learned. Sometimes dynamics come from physics: Newton's laws give us the pendulum equation $\ddot{\theta} = -(g/l)\sin\theta - (c/ml^2)\dot\theta + u/(ml^2)$ up to a few constants. Often they must be learned from data, which is the province of system identification for parametric models and of representation learning for latent ones. Part II, Chapter 3 covers system identification, and Part V covers neural architectures for learned dynamics. Between these poles lies a productive middle ground: grey-box modeling, in which physics fixes the functional form and data estimates the constants, or physics-informed networks in which known laws regularize learned components. Most successful industrial models live in this middle ground rather than at either pure extreme.

A bare dynamics model predicts motion. Agents need one more ingredient to convert prediction into preference: a model of reward, $r(s_t, a_t)$ or $r(o_t)$, scoring how good each situation or action is. With dynamics and reward together, the agent can evaluate hypothetical futures: simulate a course of action, sum the rewards along it, and compare courses. This pairing is precisely what model-based reinforcement learning exploits, and it is why reward modeling deserves first-class status rather than being treated as an afterthought bolted onto a dynamics model. The pairing also introduces a new failure surface: errors in the reward model corrupt decisions even when the dynamics are perfect, and vice versa. A planner is only as honest as the least reliable component it trusts, and disentangling which component lied after a bad outcome is a diagnostic skill worth cultivating early.

The fourth component is subtler: the model's account of agents, including the agent itself.

• Self-model: a car's world model includes its own braking capability, turning radius, and sensor blind spots. An agent that cannot predict its own body's response to commands cannot plan with its body. Robotics calls parts of this the body schema; control theory calls it the actuator model. The self-model is easy to overlook because it feels like part of the agent rather than part of the world, but from the perspective of planning, your own actuation limits are features of the environment you must predict: they determine which imagined action sequences are physically executable.
• Other-agent models: in traffic, games, and conversation, the environment contains entities that react to you. Predicting them requires modeling their goals, beliefs, and likely responses, a capacity cognitive science associates with theory of mind. Multi-agent world models remain one of the hardest open areas, and we revisit them in Part IV and Part X. The difficulty is structural, not incidental: once others react to your predictions, the fixed-target assumption underlying ordinary supervised learning dissolves, and the problem acquires a game-theoretic character.

Notice the recursion this creates: the agent's actions change the world, the changed world changes the agent's observations, and the observations update the model. A world model embedded in a closed loop is modeling a system that contains itself. This reflexivity is harmless for the pendulum but consequential for economics, ecology, and any setting where predictions influence the thing predicted. A famous illustration is traffic navigation apps: when enough drivers believe the same predicted fast route, the route stops being fast. The model's output becomes part of the system's input, and stable, accurate prediction may cease to exist at all.

The table below summarizes the anatomy and the question each component answers.

Assembling the pieces, the canonical formal object underneath most of this book is the partially observable decision process: states evolving by $p(s_{t+1} \mid s_t, a_t)$, observations emitted by $p(o_t \mid s_t)$, rewards $r$, all estimated by an agent whose internal model mirrors this structure with hats: $\hat{p}$, $\hat{r}$, and an internal state $\hat{s}_t$ updated from the observation stream. Part II, Chapter 1 gives this object its full formal treatment. Here it suffices to see that "world model" names the agent-side mirror of this structure, however crude the mirror may be. The mirroring metaphor is worth keeping: the agent never observes the true process directly, only its reflections, and the art of world modeling is building a mirror accurate enough, in the right places, to steer by.

With the anatomy in place, we can draw a distinction that sounds trivial and is anything but: the difference between predicting the future and simulating it. The confusion between these two activities is responsible for a surprising share of miscommunication in the field, partly because the word "prediction" is used loosely for both, and partly because simulation looks like repeated prediction, which suggests it should inherit prediction's guarantees. It does not, and seeing why is essential.

Prediction answers a question about the future with an estimate. Depending on need, the estimate takes different forms: a single most-likely next state (point prediction), a full conditional distribution, an expected value of some quantity, or a probability that a threshold is crossed. Prediction is typically local in time: one step, or a few. Each form of answer serves a different consumer: point predictions feed controllers that want a single target, distributions feed risk assessments, expectations feed value estimates, and tail probabilities feed safety monitors. Choosing the form is part of designing the model, not an afterthought.

Simulation produces concrete, sampled future trajectories by chaining the model through time: sample $s_{t+1}$ from the model given $(s_t, a_t)$, treat the sample as the new state, sample again, and repeat. Run the chain many times and you get an ensemble of possible futures; run it once and you get one coherent "imagined experience." Simulation is how an agent rehearses: it is the machinery behind planning by rollout, behind counterfactual reasoning ("what if I had braked?"), and behind the imagination phase of modern model-based agents like Dreamer and MuZero. The phrase "imagined experience" is chosen carefully: a single sampled rollout has the texture of an experience, a coherent sequence of states and events, which is exactly what some learning algorithms, and arguably some biological processes such as hippocampal replay, consume.

The two activities are related but not equivalent, and conflating them causes real engineering damage. Simulation is prediction applied recursively, and recursion has a cost: errors compound. If the one-step model has small error $\epsilon$, a naive hope is that $T$-step error stays near $T\epsilon$. Reality is harsher. Each predicted state is fed back in as if it were real, so the model drifts off the data distribution it was trained on, and errors interact multiplicatively through nonlinear dynamics. A model with excellent one-step accuracy can produce absurd hundred-step rollouts, a phenomenon you will watch happen numerically in the code section below. The mechanism deserves a name: feeding a prediction back as an input is called closed-loop model rollout, and the resulting mismatch between training inputs (real states) and rollout inputs (predicted states) is a form of distribution shift the model was never trained to survive.

This leads to the deepest practical lesson of the chapter, one that separates people who build world models from people who merely train sequence models.

Two corollaries follow immediately. First, open-loop prediction quality is necessary but not sufficient for closed-loop usefulness. Evaluating a world model only on held-out one-step prediction is like certifying a bridge by testing one brick. Second, optimizers exploit models. A planner searching for high-value action sequences will inevitably probe regions where the model is least constrained by data, precisely because those regions often promise unrealistic rewards. Managing this exploitation, through conservatism, ensembles, or uncertainty penalties, is a central theme of Part VII on planning and agency. The exploitation dynamic is worth internalizing as a pattern: wherever an optimizer meets an imperfect model, the optimizer finds the imperfections, because imperfections usually look like opportunity.

A related vocabulary distinction recurs throughout the literature. Open-loop means committing to an action sequence in advance and executing it without feedback. Closed-loop means re-planning or adjusting as observations arrive. Models degrade differently in the two regimes: open-loop execution suffers pure compounding error, while closed-loop use suffers repeated exposure to the model's own biases, corrected partially by fresh observations. Receding-horizon schemes such as model-predictive control sit between the extremes and are the workhorse of Part II, Chapter 4. The reason receding-horizon control works so well in practice is that it converts one long-horizon trust question into many short-horizon ones: instead of trusting the model for a hundred steps, you trust it for ten, then re-anchor on fresh reality, then trust it for ten again. Shortening the trust interval is often the single cheapest improvement available to a struggling model-based system.

Finally, note what simulation uniquely enables: counterfactuals and rare events. A predictor tells you the expected future; a simulator lets you ask what happens under interventions, under adversarial action sequences, or in the tail of the distribution. This is why safety-critical applications lean heavily on simulation, and why the fidelity of the underlying model, not just its average accuracy, becomes the binding constraint. Average accuracy is dominated by common situations, but accidents live in tails, and a model that is 99 percent accurate everywhere except the 1 percent of states where crashes occur is a dangerous model for a safety case, however impressive its aggregate metrics.

"World model" sits between two neighboring concepts with which it is constantly confused: generative models on one side and simulators on the other. Drawing both boundaries sharply will clarify a great deal of contemporary debate. Both confusions arise from real overlap: generative models and world models both model distributions over futures, and simulators and world models both answer action-conditioned queries. The boundaries lie not in what these systems compute but in what they are for and what they guarantee.

A generative model captures the distribution of data: images, audio, text, or video. Modern video generators produce startlingly plausible clips, and clips of falling objects, flowing water, and billowing smoke exhibit many regularities we associate with physical intuition. Does that make a video generator a world model?

By our operational definition, usually not yet, for three reasons.

First, action-conditioning. A video model samples $p(\text{future frames} \mid \text{past frames})$. A world model must answer $p(\text{future} \mid \text{past}, a)$: what changes because of what I do. Without an action channel, the model can extrapolate passive dynamics but cannot support decisions. Interactive generative models that accept latent actions, such as the Genie line of work, are explicitly designed to close this gap, and we study them in Part IX. The gap is deeper than adding an input slot: conditioning on actions requires training data in which actions were actually taken and their consequences observed, which means interaction data, teleoperation logs, or simulator rollouts, none of which passive internet video provides.

Second, controllability and consistency. A world model must respond coherently and persistently to interventions: if the agent picks up a cup, the cup should stay picked up in all subsequent predictions. Purely correlational video models can "forget" object permanence over long horizons because nothing in their training objective forces consistency under intervention. Causal structure, not just statistical fidelity, is at stake here, and Part IV treats causality in world models in depth. Consistency failures are subtle because they are invisible in short clips: a two-second generation can look flawless while containing objects that flicker, merge, or teleport across cuts, defects that only matter to an agent trying to maintain a coherent plan across minutes.

Third, decision sufficiency. Even an action-conditioned, consistent video model is a world model only relative to tasks whose decisions it improves. A gorgeous renderer with subtly wrong contact dynamics may produce beautiful videos while misleading a robot into crushing every object it grasps. Contact dynamics are the canonical example because they are visually subtle and mechanically decisive: a millimeter of penetration error is undetectable to the eye and catastrophic to a gripper.

The honest summary is that the boundary is a gradient, not a wall. Action-conditioned generative models trained on interaction data increasingly qualify as world models; passive ones do not. What converts a generative model into a world model is not architectural glamour but the operational test: conditioning on actions, consistency under intervention, and usefulness for control. Expect the gradient to shift over time as interactive data becomes more abundant, and expect the debate to continue, because the philosophical register asks questions the operational test cannot settle.

At the other end sits the simulator: MuJoCo, Isaac Gym, game engines, flight simulators, digital twins. A simulator certainly predicts action-conditioned futures, so is it a world model? Under the operational definition, yes: it is a world model whose dynamics were authored rather than learned. The interesting question is what distinguishes the two origins, because the differences drive methodology across this entire handbook.

Key contrasts between learned world models and engineered simulators:

• Source of dynamics: A simulator's rules are hand-coded from physical theory or game design. A learned world model's rules are estimated from data, inheriting the biases and gaps of that data.
• State access: A simulator exposes the true underlying state directly, and can be paused, inspected, and reset to arbitrary states. A learned model typically works in a latent space with no guaranteed correspondence to physical quantities, and cannot be reset to an arbitrary ground-truth configuration.
• Uncertainty: A simulator is (within its domain) confidently wrong or confidently right; it expresses no doubt. A learned model can and should represent its own uncertainty, which enables cautious exploration and risk-sensitive planning.
• Fidelity guarantees: Simulator fidelity is audited against requirements and validated by engineers. Learned model fidelity is statistical, holding only near the training distribution, and silently degrading outside it.
• Cost profile: Simulators are cheap to query but expensive to build and calibrate; learned models are expensive to train but amortize across tasks and can capture realities too messy to hand-code.

Each weakness of one is a strength of the other, which explains the two great hybrid strategies of the field. Sim-to-real uses engineered simulators as training grounds and transfers policies to reality despite the inevitable mismatch. Learning from real data builds world models where no adequate simulator exists, such as deformable objects, biology, finance, and the open web. And the two strategies converge in modern practice: learned world models trained inside simulators, simulators calibrated against real data, and foundation-scale models distilling both. Part XII on reliable world models confronts the failure modes of each approach. The convergence is worth watching as a trend: the historical split between "authored worlds" and "learned worlds" is dissolving into a continuum of calibrated hybrids, where each side patches the other's weaknesses.

It is worth naming the full spectrum explicitly, because individual systems populate every rung:

• Lookup tables and memorized transitions: a model that has only seen exactly the situation before.
• Parametric regressors: linear models, Gaussian processes, small networks fitting $(s, a) \to s'$.
• Latent dynamics models: neural encoders and recurrent transitions over learned state spaces, the architecture family of Part V.
• Action-conditioned generative models: pixel- or token-space predictors of futures under actions.
• Engineered simulators: authored dynamics with privileged state access.

Nothing about this ladder implies that higher is better. A well-fitted linear model around an equilibrium point flies aircraft and balances robots every day, while a magnificent video generator remains unable to decide anything. Fitness for purpose, again, is the only ranking that matters. Treat the ladder as a menu of complexity levels, ordered by expressive power, and choose the lowest rung that satisfies the decision-sufficiency requirement for your task. Complexity you do not need is not neutral: it costs data, compute, interpretability, and failure modes.

Let us now make every abstraction tangible with the running example. Picture a pendulum: a mass on a rod of length $l$, subject to gravity $g$ and viscous damping $c$, which you can push with a torque $u_t$. Its physical state is the pair $s_t = (\theta_t, \omega_t)$, angle and angular velocity, and its continuous-time dynamics are

Now place frosted glass between yourself and the pendulum. You do not see $\theta$ cleanly; you see a blurred, noisy reading $o_t = \theta_t + \varepsilon_t$. You never see $\omega$ at all. The frosted glass is not decoration: it is the device that forces every interesting question in this chapter to become concrete. Without it, state estimation is trivial and partial observability is a footnote. With it, you must decide what to remember, what to infer, and how much to trust each.

Walk through the anatomy with this concrete case:

• Observation: a noisy scalar, the blurred angle. Partial (velocity invisible) and noisy (frost scatters light).
• State: the pair $(\theta, \omega)$, which is Markov for the physics. Your eyes give you neither coordinate exactly.
• Dynamics: the nonlinear equation above, discretized into time steps. Deterministic here, though we will inject process noise shortly to make a point.
• Reward and agent: suppose your goal is to regulate the pendulum near its stable hanging equilibrium, so reward penalizes deviation of $\theta$ from zero and punishes large torques. As the agent, you also implicitly model your own actuator: torques above some limit simply cannot be produced.

Now trace the conceptual distinctions through the same story. Prediction: given your best current guess of $(\theta, \omega)$, estimate where the pendulum will be one step from now. Simulation: mentally chain many steps forward under a candidate torque schedule, rehearsing how quickly the pendulum settles after a disturbance before committing. Partial observability in action: because velocity is hidden, a model that predicts the next blurred angle from the last blurred angle alone must implicitly infer velocity from the trend of recent readings, and it will be systematically slower and noisier than a model that maintains an explicit internal estimate of $(\theta, \omega)$. Decision sufficiency: a model trained near the hanging equilibrium can support local regulation but says little about a swing-up trajectory through distant states. Each of these four traces corresponds to one stress test in the code below, so as you read the implementation, keep asking which conceptual claim each experiment is checking.

The following sections implement exactly this scenario. We will collect data from the true pendulum, fit a world model to it, and then stress-test the model in the three ways this chapter says matter: recursive rollout error, prediction versus simulation, and closed-loop control under model bias. The pipeline is intentionally minimal, five short stages with no hidden machinery, so that every result can be traced back to a concept rather than to an implementation detail.

Everything in this chapter so far has been conceptual. Time to make it executable. We will proceed in five stages: simulate the true pendulum and collect experience, fit a linear world model to that experience, measure how prediction error compounds over recursive rollouts, contrast point prediction with stochastic simulation, and finally close the loop by designing a controller from the learned model and seeing what model bias does to real behavior. Keep a running tally as you go: each stage instantiates one term from the definitions above, and by the end you should be able to point at a specific line of code for each clause of the operational definition.

First, the ground-truth pendulum. We integrate the nonlinear dynamics with a classical fourth-order Runge-Kutta scheme at a fixed timestep, and we gather training experience by injecting random torques from randomly sampled nearby-initial conditions. Random excitation is the system identification analogue of exploration: it keeps the dataset informative across the state region we care about. The choice of excitation range is itself a modeling decision: we sample angles within about half a radian of the hanging equilibrium because that is where our downstream local-regulation task will operate, and collecting data elsewhere would spend effort on states the controller will not visit. This is decision sufficiency shaping data collection before any model exists.

We now fit the simplest honest world model: a linear one-step predictor $\hat{s}_{t+1} = A\hat{s}_t + B u_t$, estimated by least squares on the collected transitions. Linearization is exactly the right complexity here because our data lives near the hanging equilibrium, where $\sin\theta \approx \theta$. This mirrors how classical control engineers identify linear models around operating points, a practice Part II, Chapter 3 formalizes as system identification. The regularization term added to the normal equations is tiny, but it is not cosmetic: it guards against the degenerate directions that arise when inputs are correlated, and it is the smallest possible acknowledgment of the statistical humility that larger learned models must show in far greater measure.

The heatmap makes the "locally right" claim visible. The dark valley of low error hugs the origin, exactly where the random excitation concentrated, and the bright ridges at large angles mark where the model's sine-free structure stops describing reality. Every failure mode discussed later in this chapter, compounding rollout drift and optimizer exploitation included, is a story about trajectories wandering from the dark valley into the bright ridges.

Now we interrogate the model the way this chapter insists you must: not by one-step accuracy alone, but by recursive rollout. We generate fresh test episodes unseen during training, then compare three things at every horizon: the model's one-step prediction made always from the true previous state, its recursive rollout in which each prediction feeds the next step, and a rival observation-only autoregressive model that, like our frosted-glass observer, sees only noisy angles and must limp along without velocity. The third competitor is included deliberately: it turns the abstract claim "partial observability taxes everything" into a measurable curve, and it previews the filtering machinery of Part II by showing what a model pays when it lacks an explicit state estimate.

The picture confirms the chapter's central warning quantitatively. The one-step curve stays essentially flat: anchored on truth each time, the linear model is a fine local predictor. The recursive curve climbs without bound: each small error becomes the next input, and the model wanders off the manifold where its least-squares fit is valid. Meanwhile the observation-only model trails at every horizon, paying a permanent tax for lacking velocity. If you ever evaluate a world model solely by one-step loss, this figure is the rebuttal. Read the log scale carefully: the vertical axis spans orders of magnitude, so the widening gap between the curves represents not a modest degradation but a qualitative collapse of usefulness at long horizons. Also notice the shape of the recursive curve: it accelerates before it climbs steadily, which is the signature of errors leaving the linear regime and entering the region where the model's local approximation no longer describes the true dynamics at all.

Next we illustrate the prediction-simulation distinction directly. We fix an initial state and an action sequence, then do two things with the same fitted dynamics. First, a point prediction: propagate the deterministic model forward, one crisp line. Second, a simulation: add small process noise at every step and sample many stochastic rollouts, producing a fan of imagined futures. We also run one realization of the true nonlinear system under the same noise, to see whether reality stays inside the fan. The comparison is fair by construction: both the crisp line and the fan come from the same fitted matrices, so any difference in their behavior is attributable purely to how uncertainty is represented, not to which model was used.

The widening band is the quantitative shadow of the fan chart above. Early in the rollout the ensemble members barely differ, so the model is effectively making a point prediction; by the end of the horizon the spread has grown several-fold, and the honest answer to "where will the pendulum be?" is a range, not a number. When you later meet planners that penalize model uncertainty, this curve is the quantity they consume: the further ahead a plan reaches, the wider the band, and the less any single imagined outcome should be trusted.

Our final test addresses decision sufficiency. We design a linear-quadratic regulator (LQR) around the stable hanging equilibrium, using the linearized pendulum as the design model, and we execute the resulting controller on the true nonlinear plant. Then we repeat the exercise with a deliberately biased model, one that believes torque is only 40 percent as effective as it really is, which is precisely the kind of error real identification pipelines produce when actuator gains are miscalibrated. The discretization here uses the exact matrix exponential of the augmented continuous-time system rather than a crude Euler approximation, a detail that matters: we want the only discrepancy between design model and plant to be the deliberate bias plus the nonlinearity, not an artifact of sloppy discretization.

The bar chart condenses the trajectory comparison into the single number a decision-maker cares about: the biased model's controller pays measurably more for the same task, despite both controllers eventually succeeding. This is the operational definition's final clause rendered as arithmetic. A model can pass every open-loop accuracy check and still fail here, and conversely a modest improvement in model honesty converts directly into closed-loop savings.

You now hold a complete, if miniature, world model pipeline: experience collection, model fitting, open-loop stress tests, stochastic imagination, and closed-loop deployment with an eye on model bias. Every chapter that follows scales up one or more of these five stages. When a later chapter introduces a recurrent latent dynamics network or a foundation-scale video predictor, resist the temptation to treat it as a new kind of thing: it is still this pipeline, with each stage grown more powerful and each failure mode grown more expensive.

It is worth pausing on what this tiny exercise already reveals about the difficulties ahead, because none of them disappear at scale; they change costume.

The most fundamental limitation is that compounding error and distribution shift are structural, not incidental. Our linear model drifted off its valid region within a few dozen steps, and no amount of additional data near the training region fixes that, because the failure occurs precisely where the data is absent. Modern latent world models with powerful recurrent cores push the horizon out considerably, but the qualitative phenomenon survives: recursive imagination eventually leaves the land of trustworthy predictions. The practical mitigations, training on model-generated rollouts, uncertainty-aware ensembles, and conservative planning, each trade off raw accuracy against robustness, and Part VI and Part VII treat these trades in depth. It helps to think of these mitigations as three different answers to the same question: how do we keep the optimizer inside the region where the model tells the truth? Rollout training enlarges that region, ensembles detect when we have left it, and conservatism discourages approaching its edge.

A second limitation surfaced in the closed-loop test: model bias interacts with optimization in dangerous ways. Our biased LQR merely lost some efficiency, but a planner optimizing aggressively against a biased model can do far worse, seeking out exactly the actions where the model is most flattering and least truthful. This "model exploitation" means that improving the planner without improving the model's honesty can degrade real-world performance, a paradox anyone combining strong optimizers with imperfect models must internalize early. The paradox has a corollary that surprises many practitioners: sometimes the best way to improve a system is to weaken the optimizer, for instance by adding noise to planned actions or penalizing model uncertainty, because a less exploitative planner extracts more value from an imperfect model than a greedy one does.

Third, partial observability taxes everything. Our observation-only baseline paid a permanent accuracy penalty for lacking velocity, and in richer domains the hidden state is vastly larger: intentions of other drivers, contents of unobserved shelves, thermal states of a battery. Learning representations that compress history into decision-sufficient latent states is arguably the core technical problem of modern world modeling, and it is why Part III on representing agents and worlds precedes the architecture chapters. The tax compounds with every other difficulty: a partially observable world makes exploration harder, because the agent cannot tell whether an outcome reflected its action or a hidden variable, and it makes evaluation harder, because two models can agree on observables while disagreeing about the latents that drive long-horizon divergence.

Against these limitations stands what the concept has made possible. Framing prediction as a modeling problem gave reinforcement learning a viable alternative to model-free trial and error: agents like Dreamer learn rich latent dynamics from pixels and plan inside their own imaginations, achieving sample efficiencies model-free methods cannot match. MuZero extended planning to domains where even the rules of the game are unknown, learning a model good enough to master Go, chess, and Atari from scratch. Beyond games and robotics, the same template now drives video-generation-based agents, scientific surrogate models, and digital twins. The intellectual payoff is equally large: the operational definition gives the field a shared, falsifiable standard, namely that models must earn their keep through the decisions they enable, and the next chapter turns that standard into an explicit qualification test you can apply to any candidate world model, including the ones you build yourself. Falsifiability is the quiet hero of this story: because the standard is behavioral, it can fail, and because it can fail, progress on it is measurable. Fields advance fastest where their claims can lose.

This chapter established the conceptual foundation on which the rest of the handbook rests.

• A world model is defined operationally: a representation that predicts action-conditioned consequences accurately enough to improve decisions. Philosophical traditions from Craik and Tolman enrich this with questions of representation, grounding, and acquisition.
• Every world model decomposes into observations, state, dynamics, and agent structure (rewards, self-models, other-agent models). The Markov property defines what makes a state sufficient, and partial observability is the reason internal state estimation exists at all.
• Prediction and simulation are distinct activities: prediction estimates futures, simulation samples them, and recursive simulation compounds error. Open-loop accuracy is necessary but not sufficient for closed-loop usefulness; the operative standard is decision sufficiency.
• World models sit between generative models (which lack action-conditioning, intervention-consistency, or decision relevance) and simulators (whose dynamics are authored rather than learned, with privileged state access and no expressed uncertainty). The boundary is a gradient, and fitness for purpose ranks systems better than architectural sophistication does.
• The pendulum-behind-frosted-glass example demonstrated all of these ideas in code: learned linear dynamics, compounding rollout error, the penalty of partial observability, honest uncertainty through stochastic simulation, and the real cost of model bias in a closed loop.

If the chapter has a single sentence to carry forward, it is this: judge a world model by the decisions it improves, not by the futures it forecasts. Every evaluation method, architecture choice, and research debate in the coming chapters is a variation on applying that sentence rigorously.

The immediate question is how to tell a real world model from an impressive pretender. The next chapter, The World-Model Qualification Test, builds exactly that instrument: a battery of criteria, from rollout fidelity to intervention consistency, that any candidate model must pass before it deserves the name.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about world models, their components, and how they differ from related concepts.