Taxonomies of World Models

Imagine two engineering teams asked to build a world model for the same robotic arm. The first team writes down rigid-body equations, measures the link masses, and produces a simulator whose every intermediate quantity has a physical meaning. The second team records thousands of hours of camera footage and trains a neural network that, given a frame and a motor command, generates the next frame. Both teams can answer the question "what happens if I close the gripper now?", yet their models share almost no structure. One is a handful of differential equations; the other is billions of floating-point parameters that nobody can read.

If both count as world models, and they do, then "world model" is not a single technique but a design space. This chapter maps that space. We will walk through four axes of variation that cut across nearly every system you will encounter in this handbook:

• Explicit versus implicit: whether the model's predictions can be inspected and queried directly, or whether its knowledge of dynamics hides inside something else, like a policy.
• State representation: whether prediction happens over pixels, discrete tokens, compressed latent vectors, objects, or symbols.
• Deterministic versus stochastic dynamics: whether the model commits to a single future or distributes probability over many.
• Task-specific versus general-purpose: whether the model serves one environment and one objective, or many.

These axes are largely independent. You can combine them freely, which is why the field contains hundreds of named architectures that reduce to a handful of positions in this design space. By the end of the chapter you should be able to place any new paper's model on these axes within a few minutes of reading, and, more importantly, reason about which position your own application demands.

Throughout, keep the distinction from Chapter 2 in mind: a world model is a predictor of environmental consequences, and its value is realized only when an agent uses those predictions. Several taxonomic choices below look like modeling choices but are decision-making choices in disguise. When you decide whether your model predicts pixels or latent vectors, you are really deciding what your planner will be able to imagine. When you decide between deterministic and stochastic dynamics, you are really deciding what kinds of risk your controller will be able to see. Keeping this lens active will save you from optimizing modeling details that have no bearing on the decisions your agent ultimately makes.

Before diving into the axes, it is worth asking why a taxonomy earns its keep. Three reasons stand out.

First, taxonomies expose tradeoffs. Every choice along each axis buys something and costs something. Pixel-level models preserve maximal information but make dynamics learning brutally hard. Latent models make dynamics easy but risk discarding exactly the information your task needs. Naming the axes turns vague anxiety ("will my representation lose something important?") into concrete engineering questions. Once you know that the question is "where does abstraction happen, and what does it cost?", you can inspect a candidate architecture and ask precisely which information its bottleneck preserves, which it destroys, and whether the destroyed part matters for your reward function, your safety constraints, or your evaluation metrics.

Second, taxonomies prevent false debates. When one researcher says "world models need object structure" and another says "end-to-end latent models work fine," they are often arguing about different points on the state axis while tacitly assuming different tasks, different data budgets, and different control regimes. A shared vocabulary makes the actual disagreement visible. Many celebrated controversies in the field dissolve once you notice that the participants were optimizing different quantities: one cared about sample efficiency in a low-data regime, another about asymptotic performance with unlimited data, and a third about interpretability for human operators. None of these goals is wrong, but a model cannot be ranked without saying which goal governs.

Third, taxonomies connect modern systems to classical ones. A latent dynamics model trained with gradient descent and a state-space model identified with subspace methods in Part II are answers to the same question posed with different tools. Seeing the continuity helps you import sixty years of theory, including observability, controllability, and identifiability, into modern settings where those words rarely appear. A learned encoder mapping observations to latent vectors is doing work that classical estimation theory would describe as constructing an observer, and questions like "can two distinct physical states produce identical observation histories?" apply to both with equal force. Similarly, the stochastic models we discuss below descend directly from the Kalman filter's treatment of process noise, and the compounding-error analysis we revisit descends from classical stability theory for closed-loop control under model mismatch.

A fourth reason deserves mention alongside these three: taxonomies separate kinds of model quality that practitioners frequently conflate. A model can be predictively accurate yet representationally poor, meaning its outputs match reality while its internal variables support no useful reasoning. It can be well calibrated yet decision-useless, meaning its uncertainty estimates are honest but too coarse to act on. And it can excel at one-step fidelity while failing catastrophically at multi-step rollouts. Throughout this chapter we will flag which axis affects which kind of quality, because confusing them leads to the most common evaluation mistakes in the field.

With that motivation, let us take the axes one at a time.

The first and most fundamental question about any world model is: can you ask it a question? An explicit model exposes a prediction interface. Given a hypothetical state and action, it returns a predicted outcome, and you can chain such queries into arbitrary rollouts. An implicit model contains the same kind of knowledge, but fused into a larger computation, so that no component in isolation answers "what happens next?"

Most of this handbook concerns explicit models, because explicitness is what enables the classic uses: planning by lookahead, model-based data augmentation, counterfactual reasoning, and analysis. Classical examples include the analytical simulators of aerospace engineering, the identified state-space models of Part II, and learned forward models like the ones we will build later in this chapter. The defining property is composability: because each prediction returns a state in the same format the model consumes, you can recurse. One-step accuracy becomes, at least in principle, multi-step foresight.

This composability is exactly what made classical model-predictive control possible decades before anyone spoke of world models. An MPC controller at each time step asks the model a short sequence of hypothetical questions: "if I apply torque $u_1$ now, then $u_2$ next step, and so on, where does the plant end up?" The answers form a predicted trajectory, the controller scores it against a cost function, applies the first action, and repeats. Every ingredient of that loop, the rollout, the scoring, the replanning, presupposes an explicit interface. The same is true of classical observers such as the Luenberger observer and the Kalman filter: both maintain an explicit internal estimate of the state and update it using an explicit model of how states evolve. Modern learned world models inherit this entire workflow; only the source of the transition function changes.

Explicitness also carries obligations. An explicit model must decide what its outputs are: pixels, latents, objects, or symbols. That decision is the subject of the next section. And an explicit model can be exploited, meaning a controller can steer it into regions where its predictions are confidently wrong. We treat that failure mode as a first-class concern throughout Part VII and Part XII. The exploitation problem has a classical analogue too: aggressive MPC controllers have always been able to drive a nominal model outside the region where its linearization was valid, and the field's answer, robust and tube-based MPC, anticipates the uncertainty-aware techniques we develop later.

Implicit models are more common than the term suggests. Consider three examples.

• Model-free value functions. A critic trained by temporal-difference learning must, in some sense, know the dynamics: its estimates change in exactly the way the environment's transition distribution dictates. But you cannot query it with a hypothetical state-action pair and receive a predicted next state. The dynamics knowledge is real yet inaccessible. You can observe that the critic assigns high value to a state, but you cannot ask it why in terms of future events, nor redirect it to evaluate a hypothetical intervention the environment never produced during training.
• Successor features compress the transition structure of a Markov reward process into a quantity that supports rapid policy evaluation, again without exposing raw next-state predictions. They encode expected future feature occupancy, which is a summary of dynamics filtered through a discount factor, so the compression is principled but lossy in a direction chosen by the discount, not by the user.
• MuZero (Schrittwieser et al., 2020) is the sharpest modern example. Its "model" operates in a latent space and predicts only three things the agent cares about: reward, value, and policy. It never reconstructs observations. Planning works by searching over imagined latent trajectories, so the model is explicit for decisions while remaining implicit about appearance.

That last phrase suggests the useful refinement: implicitness is not binary but scoped. MuZero's model is explicit about decision-relevant quantities and implicit about everything else. Whether that scope is a strength or a weakness depends entirely on your use case. If your sole consumer is a search procedure maximizing reward, MuZero's scope is close to optimal: it spends every unit of capacity on quantities the planner reads. If any other consumer exists, a debugging engineer, a verification tool, a downstream task with a different reward, the missing scope becomes a liability, because there is no way to interrogate the model about anything beyond its three trained heads.

The tradeoff cuts both ways. Implicit models avoid paying for reconstruction of irrelevant detail, which is why MuZero-style models can be easier to train than reconstruction-based ones: there is no loss term tempting the representation to spend capacity modeling visually salient but decision-irrelevant texture. Explicit models, in exchange for that cost, support uses that implicit models cannot: debugging, verification, transfer of the model alone to new tasks, and the counterfactual queries that safety analysis in Part XII requires. Transfer deserves emphasis because it is easy to overlook: an explicit dynamics model trained on one reward can be re-plugged into a new reward immediately, whereas an implicit model whose value head was shaped by the old reward carries assumptions about objectives baked into its weights, and retraining is usually unavoidable.

A practical rule of thumb: be explicit about whatever you will query, and implicit about whatever you will not. If your agent plans by imagining observations, model observations. If it plans by imagining values, model values. Trouble begins when you guess the wrong scope, for example building a beautiful video predictor for an agent that only ever needed two bits of contact information. The inverse error is just as real: training a purely value-scoped model and then discovering mid-project that you needed to audit its imagination, diagnose a failure, or satisfy a regulator who wants to see what the agent believes will happen. Scope decisions are cheap to make early and expensive to reverse late, which is why we treat them as a first-order design choice rather than an implementation detail.

Every explicit model must commit to a format for the states it predicts. This choice, more than any architectural detail, determines what the model can learn easily, what it will struggle with, and how it can be used. We distinguish five broad families. They form a rough spectrum from raw sensation to abstract structure:

• Pixel-space models predict raw sensory signals directly.
• Token-based models discretize perception into symbols and predict symbol sequences.
• Latent-state models predict in continuous compressed vectors learned end to end.
• Object-centric models predict over persistent entities with attributes and relations.
• Symbolic models predict over discrete, human-interpretable relational states.

It helps to hold two questions in mind as we walk this spectrum. First, when does abstraction happen: before learning dynamics, during it, or not at all? Second, who chooses the abstraction: the data, a fixed pretrained module, an architectural prior, or a human designer? Every family below is an answer to both questions, and most practical disagreements about representations reduce to disagreements about those answers. The spectrum below previews the tradeoff we will keep returning to: the further right a family sits, the earlier abstraction happens, the easier dynamics learning becomes, and the more the representation bets on assumptions that may not hold for your task.

A pixel-space (more generally, observation-space) model learns $p(o_{t+1} \mid o_{1:t}, a_{1:t})$ directly over raw sensory streams: images, audio, proprioceptive readings. Video prediction models are the canonical case. The appeal is obvious: no representation decisions are made for the learner, so nothing task-relevant can be thrown away by a premature bottleneck. Whatever the downstream task needs, if it is visible in the data, the model can in principle carry it. This guarantee is worth taking seriously in domains where nobody knows in advance what will matter: a general-purpose robot operating in unstructured homes cannot enumerate the features its future tasks will require, and pixel-level prediction is the only representation choice that makes no bet on the answer.

The cost is that dynamics in pixel space are violently nonlinear and mostly invariant. When an object moves one pixel to the right, almost every pixel value stays identical, a few change dramatically, and the meaning of the change depends on scene content that the pixel grid does not encode. A function class must be extraordinarily flexible to represent "the cup moved" as a pixel-to-pixel map, and even then it represents it as an incidental correlation rather than a structured fact. Pixel-space models therefore tend to need enormous data and capacity, and their rollouts blur: averaging over uncertain futures in pixel space produces the smeared frames familiar from early video prediction work. That blurring is not merely cosmetic. It is the conditional mean made visible, the same pathology we examine under deterministic dynamics below, and it means a naive pixel-space model can look excellent on per-frame metrics while producing imaginations no planner could trust.

Pixel-space prediction remains valuable in two regimes. First, as a pretraining signal: predicting missing patches or future frames forces a network to absorb intuitive physics that transfers to downstream control. Here the pixel model is scaffolding rather than the final artifact: its weights seed an encoder that later serves a more abstract model, and the expensive pixel-level knowledge is distilled rather than queried. Second, as an interface: when humans must inspect or interact with the model's imagination, observations are the currency they understand. A research team diagnosing why a latent-world-model agent failed can gain more from watching ten decoded imagined frames than from reading a thousand latent vector norms.

Token-based models insert a discretization step between sensation and dynamics. A learned tokenizer, typically a vector-quantized variational autoencoder in the lineage of VQ-VAE, maps each observation to a short sequence of discrete codes drawn from a learned codebook. The world model is then a sequence model, frequently a transformer, over these code sequences, conditioned on actions. Conceptually, the tokenizer plays the role that quantization and symbol grounding played in classical perception pipelines, except that the vocabulary is discovered from data rather than designed by hand.

Discretization buys surprising advantages. Categorical distributions are easy for maximum-likelihood training to represent sharply, avoiding the blurry averages that plague Gaussian image models. Where a Gaussian decoder faced with two plausible futures must output their average, a categorical over tokens can place half its mass on each, keeping both futures crisp and recoverable. Sequence modeling over tokens inherits the mature toolkit of language modeling: stable scaling laws, teacher forcing, and architectures tuned over years of transformer development. Systems such as IRIS (Micheli et al., 2023) showed that a transformer over image tokens, combined with a policy over the same tokens, can reach strong sample efficiency on benchmark control tasks purely from pixels.

The costs mirror the benefits. Quantization is lossy in an uncontrolled way: the codebook decides what detail survives, and two perceptually distinct states may collapse to identical tokens, destroying distinctions a task needs. Because the codebook is learned before or alongside the dynamics, nothing in its training objective guarantees that the surviving distinctions are the ones your controller requires. Token boundaries also interact awkwardly with continuity: a smoothly moving object produces token churn that the dynamics model must smooth over. Finally, planning through a discrete imagination introduces gradient pathologies, since you cannot differentiate through a categorical sample without tricks. Search-based planners sidestep this by treating the token model as a simulator to query rather than a function to differentiate, which is one reason token-based world models pair naturally with sampling-based planning rather than gradient-based trajectory optimization.

Latent-state models, the workhorses of modern model-based reinforcement learning, learn a continuous encoding $z_t = e(o_{\le t})$ together with dynamics $p(z_{t+1} \mid z_t, a_t)$, training both jointly so that the latent space becomes useful for prediction. The World Models architecture of Ha and Schmidhuber (2018) established the template, a vision encoder, a recurrent dynamics model, and a controller trained inside the dream, and the Dreamer series (Hafner et al., 2020 onward) refined it into a family of state-of-the-art agents.

The central promise of latent models is that the network can discover the state variables that make dynamics simple. Recall from Chapter 2 that a good state renders the future conditionally independent of the past. A well-trained latent model approximates exactly that: its hidden state is a running belief about the aspects of the world that matter for predicting what comes next. Readers familiar with classical filtering will recognize this as the learned cousin of the Kalman filter's state estimate: in both cases the agent maintains a compressed summary of its observation history sufficient for forward prediction, and in both cases the quality of that summary determines everything downstream. The difference is that the Kalman filter's sufficiency was guaranteed by linear-Gaussian assumptions, while the latent model's sufficiency is only as good as its training losses and data coverage.

Because the space is continuous and differentiable, gradients flow through multi-step rollouts during training, and planners can optimize action sequences by gradient descent through the imagination. This differentiability is a structural advantage: it converts planning from a combinatorial search into a continuous optimization, which is dramatically cheaper when the imagination is trustworthy enough for gradients to point in useful directions.

The corresponding risk is that "useful for prediction" is measured against the training distribution and the training losses. A latent model can achieve excellent one-step prediction while quietly discarding information that a new task, a new reward, or a rare event will suddenly need. This is the problem of control-sufficient state, and it is central to state abstraction theory, which Part III develops formally. For now, hold onto the asymmetry: pixel models risk drowning in irrelevant detail, latent models risk starving relevant detail, and neither failure announces itself in the training loss. Both failures are invisible to the metric most teams watch first, which is why the evaluation practices of Part XI insist on testing representations against perturbations and tasks the training objective never mentioned.

Object-centric models bet that the right intermediate representation is neither pixels nor unstructured vectors but entities: persistent things with attributes (position, velocity, color) and relations (contact, containment, support). Slot-based architectures such as slot attention (Locatello et al., 2020) decompose a scene into a set of vectors, one per object, and dynamics models then predict how each slot evolves, often with interaction terms between slots. The bet has deep roots: physics engines, scene graphs in computer graphics, and relational calculi in classical AI all rest on the intuition that the world is naturally described as things undergoing local interactions, and that a learner given this description faces a vastly simpler induction problem.

The payoff is compositional generalization. If the model represents the world as $n$ objects obeying local interaction rules, then a scene with a novel arrangement of familiar objects, or even a slightly larger number of objects, remains in-distribution in a way that no flat representation guarantees. Physical reasoning, multi-object manipulation, and scene editing benefit visibly from this structure. Neural physics engines and the neuro-symbolic literature build directly on this bet. In classical terms, object-centric structure is an inductive bias purchased up front: it constrains the hypothesis class so drastically that far less data is needed, provided the constraint matches reality.

The difficulty is segmentation itself. Deciding what counts as an object is easy for billiard balls and brutal for deformables, fluids, granular media, and cluttered household scenes. Is a rope one object or infinitely many? Does a pile of sand have parts, and if so, where are their boundaries? Slot-based methods also impose a fixed slot count and struggle when objects appear, disappear, or merge. Object-centric modeling is therefore best understood as a strong prior that pays off enormously when its assumptions hold and misfires when they do not. The engineering pattern that works in practice is hybrid: let a slot-based module handle the rigid, well-segmented portion of the scene while a latent or pixel pathway absorbs everything the slots fail to capture.

At the abstract end of the spectrum sit symbolic models: states are discrete propositions or relational atoms (on(block_a, table)), and dynamics are rules, as in the planning languages classical AI used for decades. These models offer perfect compositionality, exact multi-step inference, and human-auditable structure. Their historical weakness was grounding: mapping raw perception to symbols reliably, and recovering when the mapping is wrong. A symbolic planner is only as good as its perceptual front end, and classical AI spent decades discovering that the last meter from pixels to predicates resists hand-engineering.

The contemporary picture is hybrid rather than adversarial. Learned perception proposes symbols; symbolic dynamics reasons over them; execution failures feed back to repair the grounding. Large language models have accelerated this trend by providing broad prior knowledge over symbols and plans. For world modeling specifically, symbolic structure tends to enter as a scaffold around a learned model, constraining its imagination to plausible relational configurations, rather than as the whole system. A typical modern stack might run a latent world model for continuous control while a symbolic layer tracks which high-level conditions hold, plans over long horizons where numeric precision is irrelevant, and flags when the learned model's imagination violates a relational invariant such as "a grasped object remains attached until released."

Table state-families summarizes the tradeoffs. Treat the rows as positions on a dial rather than mutually exclusive camps; production systems frequently combine several, for example a latent model whose decoder emits object slots.

Two cross-cutting remarks help you use this table well. First, the axis is really about where abstraction happens: nowhere (pixels), by a fixed pretrained module (tokens), jointly with dynamics (latents), by an inductive bias (objects), or by hand plus a grounding layer (symbols). Earlier abstraction means cheaper dynamics learning but greater risk of losing what matters. Second, the right position correlates strongly with how the model will be consumed, which is why we flagged the explicit/implicit distinction first. A model imagined over by a search procedure wants low-dimensional, differentiable latents. A model inspected by humans wants decodable outputs. A model serving a symbolic planner wants entities and relations.

Notice that these consumption patterns map onto the distinctions between predictive fidelity, representation quality, and decision usefulness drawn earlier. A pixel model maximizes predictive fidelity in the sense of matching observations, but its representation may support no compact reasoning. A latent model optimizes representation quality for the losses it was trained on, which may or may not align with decision usefulness for your task. Only by asking who consumes the model, and for what, can you decide which of these qualities to prioritize, and the table is best read as a menu of priorities rather than a leaderboard.

We will return to the question of what makes a latent state sufficient for control in Part III, and to the concrete architectures that implement each family in Part V.

Independently of the state format, every model must decide how to represent the spread of possible futures. The world is noisy: sensors jitter, contacts slip, other agents act unpredictably. The taxonomy here separates models that output one future from models that output a distribution. This axis is orthogonal to the state axis in principle, though in practice certain combinations dominate: pixel and latent models appear in both deterministic and stochastic forms, while symbolic models almost always commit to single outcomes because their rule-based semantics leave little room for graded belief.

A deterministic model computes $\hat{s}_{t+1} = f_\theta(s_t, a_t)$. Training is typically squared-error regression, inference is a single forward pass, and rollouts are cheap and differentiable. Most of the systems in this handbook, including the one we build below, are deterministic. Determinism is not a lazy default: for many controlled systems, especially those with strong feedback stabilizing the dynamics, the spread of plausible futures at the timescales that matter is small enough that a single committed trajectory is an adequate summary, and the computational savings are substantial.

The blind spot appears whenever the true transition is stochastic. Squared-error training drives a deterministic model toward the conditional mean of the next state. When the future is unimodal, the conditional mean is a fine summary. When it is multimodal, the mean is a fiction: a state that lies between two plausible outcomes and may itself be physically impossible. A deterministic model of a car approaching a fork does not predict "left or right"; it predicts "into the median." The figure below makes the pathology concrete with a minimal bimodal transition: the conditional mean sits in a valley between the two modes, a value the environment essentially never produces, yet squared-error regression commits to exactly that value.

Worse, during multi-step rollouts the model treats its own fictional outputs as facts, committing to a single imagined future with no indication of how fragile it is. Nothing in the model's output distinguishes a prediction it is sure of from one it is guessing at, so a downstream planner weights them equally.

This failure has a subtle training-time cousin. Under noise, regression pushes the model to predict expectations, and a controller optimizing against those expectations systematically misestimates the value of risky options, a phenomenon related to the optimism and exploration pathologies we examine in Part VII. The mechanism is worth spelling out because it is counterintuitive: if action A yields +10 or -10 with equal probability and action B yields 0 for certain, the conditional-mean model evaluates both as 0, erasing the difference entirely. A risk-seeking planner loses the upside information; a risk-averse planner loses the downside warning. Either way, the information needed to distinguish the actions was destroyed at training time, not at decision time, which makes it unrecoverable by better planning on top of the same model.

Stochastic models instead learn a conditional distribution $p_\theta(s_{t+1} \mid s_t, a_t)$ and answer queries by sampling. There are several standard mechanisms, differing in what uncertainty they capture:

• Explicit distribution heads: the network outputs the parameters of a chosen family, such as a Gaussian mean and variance, a mixture of Gaussians for multimodality, or a categorical over discrete outcomes. Sampling is direct; likelihoods are computable. The choice of family is itself a modeling commitment: Gaussians assume unimodal spread, mixtures restore multimodality at the price of harder optimization, and categorical heads suit inherently discrete outcomes.
• Latent noise: the model injects randomness into its hidden state at every step, as Dreamer does with its stochastic latent variables, so that diversity accumulates naturally over rollouts. Rather than predicting a distribution over next states directly, the model predicts a distribution over a latent variable that modulates the transition, which keeps individual samples sharp while allowing the ensemble of samples to fan out.
• Deep ensembles: several independently trained deterministic models vote. Their disagreement is a practical proxy for epistemic uncertainty, the uncertainty coming from limited data, though it says little about inherent randomness. Ensembles are popular because they require no change to the base model, only repeated training, and because their disagreement tends to grow in exactly the regions where training data was sparse.
• Generative samplers: diffusion or flow models conditioned on history and action produce high-fidelity samples of complex futures such as images, at the cost of expensive iterative sampling. These are the current frontier for stochastic imagination over high-dimensional observations, trading compute at inference time for expressive power that parametric families cannot match.

Sampling-based models can do things deterministic ones cannot. Planners can propagate uncertainty through rollouts and choose actions that are robust across futures rather than optimal in an average fiction. Evaluation can ask not just "was the prediction accurate?" but "was the model's confidence calibrated?", which separates predictive fidelity from uncertainty quality, two properties that neither implies the other. And rare-event reasoning becomes possible: you can sample until the dangerous branch appears, instead of never seeing it at all. For safety-critical applications this last capability is often the decisive argument, since the events that matter most are precisely the ones a conditional-mean model smooths away.

The costs are real. Stochastic rollouts require many samples to estimate anything, multiplying compute. Training with likelihood objectives can be unstable, especially with variance parameters that can collapse or explode. And calibration is notoriously difficult: a model can be stochastic in form while badly miscalibrated in fact, spreading probability in ways that match neither the data nor honest ignorance. Overconfident stochastic models are arguably worse than honest deterministic ones, because they lend an air of statistical rigor to guesses, and downstream consumers tend to trust outputs wearing the costume of probability. Calibration assessment therefore belongs in any serious evaluation protocol, and Part XI treats it as a first-class metric rather than an afterthought.

Chapter 3 introduced the qualification test for world models, and one of its central lessons bears repeating here because the deterministic/stochastic choice interacts with it directly. Multi-step prediction error compounds: each predicted state becomes the input to the next prediction, so small one-step errors feed on themselves. Deterministic models compound toward a single confident wrong trajectory. Stochastic models compound toward a widening cloud, which is more honest but also harder to plan with, since the cloud eventually swallows any signal about distant consequences. Neither choice eliminates compounding; they only determine its shape. Mitigation strategies, such as short-horizon training, closed-loop fine-tuning, and uncertainty-aware replanning, occupy much of Part VI, Part VII, and Part XII.

There is a classical echo here worth hearing. Feedback control has always been the ultimate defense against model error: rather than demanding a perfect open-loop prediction over a long horizon, classical schemes re-predict at every step from freshly measured state, so errors are corrected before they accumulate. Model-predictive control formalizes this as receding-horizon planning, and the same logic transfers unchanged to learned world models. When you see a learned model's long-horizon rollouts degrade, the practical response inherited from control theory is not necessarily a better model but a shorter horizon plus faster replanning, letting measurement close the loop that imagination cannot hold open alone.

The final axis concerns scope. Most world models built to date are task-specific: trained on data from one environment, often with that environment's reward signal entangled in the training objective, and discarded when the task changes. Dreamer agents are trained per game; identified dynamics models describe one robot arm; the pendulum model we build below describes one pendulum. Task specificity is not a defect so much as a consequence of economics: collecting broad interaction data is expensive, and if your decision problem lives in one environment, generality beyond it is wasted capacity.

General-purpose world models aim to break that coupling. The ambition, visible across recent research lines, is a single model trained on internet-scale video and interaction data that captures broadly reusable dynamics: objects persist, liquids pour, people walk, gravity pulls downward. Systems in this spirit include large video generation models such as Sora, interactive world generators such as Genie (Bruce et al., 2024), and industrial efforts such as NVIDIA's Cosmos, which explicitly targets physical AI applications. LeCun's JEPA proposal argues that the path runs through latent prediction of abstract representations rather than pixel generation. Note how each of these programs occupies a different position on the earlier axes: Sora-style models are explicit and pixel-scale, Genie is explicit over latent action-conditioned dynamics, and JEPA advocates latents with deliberately suppressed pixel detail. The generality debate is thus inseparable from the representation debate, and progress on one axis reshapes the options on the other.

Moving rightward along this axis changes far more than dataset size:

• Action conditioning becomes the hard part. Passive video teaches how the world evolves on its own; agents need "what happens if I do X," and internet video rarely contains reliable action labels. Inferring or annotating actions, or learning latent action spaces as Genie does, is an active research frontier we revisit in Part IX. The difficulty is deeper than labeling: much of what makes video useful for prediction is precisely that it was produced by someone's unrecorded intentions, and recovering those intentions from pixels alone is an inverse problem with no unique solution.
• Evaluation changes character. A task-specific model is judged by the performance of the agent it powers. A general-purpose model needs benchmarks for physical plausibility, controllability, and consistency across contexts, the concerns of Part XI. There is currently no consensus equivalent of "win rate" for general world models, which slows progress in ways that raw compute cannot fix.
• Failure modes scale up. A task-specific model that hallucinates wastes one training run. A general-purpose model that hallucinates convincingly, at scale, becomes an infrastructure risk wherever downstream agents trust it. Convincingness and correctness decouple at scale: a model trained to produce plausible video has every incentive to render physically impossible events beautifully, and consumers who judge by appearance will be misled systematically rather than occasionally.

It is worth resisting the temptation to treat this axis as a progress ladder with general-purpose models at the top. Task-specific models remain the right answer whenever you have a fixed environment, a modest data budget, and a concrete decision problem. Generality is expensive, and its value materializes only when amortized across many downstream uses. The correct framing is amortization: a general-purpose model is a fixed cost spread over an uncertain number of future tasks, and it wins exactly when that number turns out large. Betting on it is a strategy decision, not a scientific inevitability.

Taxonomies become concrete only when you watch the choices play out on real data. So we will spend the rest of the chapter doing exactly that, with a deliberately small system where every quantity can be inspected. Smallness is the point: with sixteen observation dimensions and a two-dimensional true state, we can compute ground truth exactly, plot every trajectory, and attribute every discrepancy to a specific modeling choice rather than to scale, data volume, or infrastructure.

The environment is a damped pendulum with a torque input, simulated with simple Euler integration and a small amount of process noise on the angular velocity. Its true state is two-dimensional: angle $\theta$ and angular velocity $\dot{\theta}$. The physics are elementary:

where $u_t$ is the applied torque and $\varepsilon_t$ is Gaussian process noise. The agent never sees $(\theta, \dot{\theta})$ directly. Instead it observes a sixteen-dimensional "pixel" vector produced by a fixed nonlinear sensor: a random linear projection of the sufficient features $(\sin\theta, \cos\theta, \dot{\theta})$, passed through a $\tanh$ nonlinearity. The sensor is invertible in principle, so no information is lost, but the observation coordinates are a tangled mixture of the physical state. This setup isolates exactly the phenomenon the state axis is about: the difficulty lies not in missing information but in coordinates, and the experiment will show how much coordinate choice alone is worth.

Into this arena we send three contestants, one from each of the taxonomic positions we just discussed:

• Contestant A, the observation-space linear model, fits linear dynamics directly in the sixteen-dimensional observation space. This is the pixel-space philosophy taken to its cheapest conclusion: no representation learning, just regression where the data lands.
• Contestant B, the learned latent-state model, trains a small encoder to recover the three dynamics-relevant features from nonlinear observations, then advances those features with the known pendulum transition. The supervised state target is a deliberate teaching convenience: it isolates state-estimation error from transition-model error.
• Contestant C, the oracle feature model, receives the true sufficient features $(\sin\theta, \cos\theta, \dot{\theta})$ and applies the same structured transition. No realistic agent receives this state for free, but the contestant provides a noise-limited reference.

We evaluate all three the same way: fit or initialize from thirty training episodes, then roll out sixty steps from ten held-out initial states with held-out action sequences, measuring the root-mean-square error of the predicted angle against ground truth at every horizon. Finally, we estimate process-noise scale from Contestant C's one-step residuals and build a stochastic variant.

Before running the comparison, we expect Contestant A to accumulate structural error because a linear map in observation coordinates cannot express the nonlinear sensor dynamics. Contestant B should approach the oracle when its encoder estimates state accurately, while Contestant C should expose the irreducible error caused by process noise. The comparison is intentionally diagnostic rather than a fair architecture benchmark, because B and C receive a known transition to isolate the cost of state representation.

We begin with the shared substrate: the simulator, the dataset, and the nonlinear sensor. Everything downstream consumes these arrays.

All computation is done, and what follows are three views of the same underlying arrays.

The headline result is the growth of multi-step error with horizon, plotted on a logarithmic scale so that the early errors remain visible beside the much larger long-horizon errors.

Read this figure as a portrait of honest ignorance. The deterministic model draws one line and asserts it. The stochastic model draws a cloud whose width grows with horizon, reflecting accumulating uncertainty, and the true trajectories travel inside the cloud rather than along its center. Nothing about the cloud tells a planner what to do; converting sampled futures into risk-sensitive decisions is the business of Part VII. But the cloud is the raw material that risk-sensitive planning consumes, and a deterministic model cannot manufacture it after the fact. Note also what the figure does not show: any guarantee that the cloud's width is correct. With only five reference trajectories overlaid, we can check bracketing informally, but rigorous calibration assessment would demand many more trials and a proper scoring rule, which is exactly the methodology Part XI develops.

It is worth being precise about what this toy demonstrates and what it does not. The feature-space model enjoys an unfair advantage we handed it: access to the sufficient statistics. The learned encoder also receives supervised state targets, a privileged diagnostic that isolates state-estimation error from transition-model error rather than a general recipe for representation learning. Part III develops methods that try to recover useful state from observations without such labels, using reconstruction, self-supervised dynamics, rewards, or combinations of objectives. Token-based and object-centered variants make still different tradeoffs. The pendulum fixes the shape of the argument, not the engineering frontier. What transfers from toy to frontier is the logic: identify which taxonomic position a design occupies, predict its characteristic failure from that position, and check the prediction against rollout behavior rather than one-step accuracy.

The taxonomic view has real limits, and pretending otherwise leads to poor research taste. The five-way state axis is a simplification of a continuum: production systems routinely blend families, running a latent core with a token interface and object-slot decoders, and the labels describe components more often than whole systems. The explicit/implicit distinction blurs further in agents that distill an explicit planner into an implicit policy, or distill an implicit value function back into an explicit model for analysis. Treat every placement in this chapter as a statement about the dominant interface, not a metaphysical commitment.

There is also a deeper worry: axes that describe architecture may not describe capability. Two models with identical taxonomic positions can differ enormously in what they know, and the same architecture can express radically different physics depending on data and training. This is why the handbook devotes Part XI and Part XII to evaluation and reliability rather than treating architecture as destiny. A taxonomy tells you what questions to ask of a model; only experiments tell you the answers. Concretely, knowing that a model is a stochastic latent model tells you to check its calibration and its rollout compounding, but it tells you nothing about whether its latents distinguish the situations your safety case cares about, and no amount of taxonomic reasoning substitutes for probing that directly.

The taxonomic framing had substantial practical value. It gave the field a shared vocabulary during a period of rapid architectural change, letting results transfer across communities that did not know they were working on the same problem: the Dreamer lineage's latent rollouts, MuZero's decision-scoped implicitness, video foundation models' pixel-scale priors, and neuro-symbolic systems' relational scaffolds are all legible as positions on the axes above, with their tradeoffs largely predictable from position alone. For practitioners, the axes function as a design checklist. Before building, you can now ask: what will query this model, in what format, over how long a horizon, under how much noise, and across how many tasks? Each question eliminates half the design space, and the chapters ahead assume you can answer them.

This chapter mapped the world-model design space along four largely independent axes:

• Explicit versus implicit: explicit models expose queryable predictions and enable planning, analysis, and counterfactuals; implicit models embed dynamics in policies, values, or decision-scoped predictors like MuZero, trading inspectability for freedom from irrelevant reconstruction.
• State representation: pixel, token, latent, object, and symbolic formats trade abstraction timing against information preservation. Pixels drown in irrelevant detail, latents risk silently discarding relevant detail, objects and symbols buy compositionality at the price of strong assumptions.
• Deterministic versus stochastic dynamics: deterministic models regress toward conditional means and imagine single confident futures; stochastic models represent spread through distribution heads, latent noise, ensembles, or generative samplers, distinguishing aleatoric from epistemic uncertainty.
• Task-specific versus general-purpose: scope changes action conditioning, evaluation, and failure modes, not merely dataset size.

The pendulum experiment made the stakes concrete: with data, actions, and evaluation held fixed, a dynamics-compatible latent representation reduced sixty-step error by roughly a factor of ten relative to a linear observation-space baseline, and the deterministic/stochastic choice determined whether uncertainty was visible at all.

These categories did not emerge from a vacuum. The ideas behind them, feedback, state, prediction, and the strange loop of an organism modeling its own environment, accumulated over more than a century of cybernetics, control theory, cognitive science, and machine learning. The next chapter traces that intellectual and cognitive history, because knowing where these ideas came from explains a great deal about why today's systems look the way they do.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about the taxonomies of world models.