Taxonomies of World Models › Taxonomies of World Models

Imagine two engineering teams asked to build a world model for the same robotic arm. The first team writes down rigid-body equations, measures the link masses, and produces a simulator whose every intermediate quantity has a physical meaning. The second team records thousands of hours of camera footage and trains a neural network that, given a frame and a motor command, generates the next frame. Both teams can answer the question "what happens if I close the gripper now?", yet their models share almost no structure. One is a handful of differential equations; the other is billions of floating-point parameters that nobody can read.

If both count as world models, and they do, then "world model" is not a single technique but a design space. This chapter maps that space. We will walk through four axes of variation that cut across nearly every system you will encounter in this handbook:

• Explicit versus implicit: whether the model's predictions can be inspected and queried directly, or whether its knowledge of dynamics hides inside something else, like a policy.
• State representation: whether prediction happens over pixels, discrete tokens, compressed latent vectors, objects, or symbols.
• Deterministic versus stochastic dynamics: whether the model commits to a single future or distributes probability over many.
• Task-specific versus general-purpose: whether the model serves one environment and one objective, or many.

These axes are largely independent. You can combine them freely, which is why the field contains hundreds of named architectures that reduce to a handful of positions in this design space. By the end of the chapter you should be able to place any new paper's model on these axes within a few minutes of reading, and, more importantly, reason about which position your own application demands.

Throughout, keep the distinction from Chapter 2 in mind: a world model is a predictor of environmental consequences, and its value is realized only when an agent uses those predictions. Several taxonomic choices below look like modeling choices but are decision-making choices in disguise. When you decide whether your model predicts pixels or latent vectors, you are really deciding what your planner will be able to imagine. When you decide between deterministic and stochastic dynamics, you are really deciding what kinds of risk your controller will be able to see. Keeping this lens active will save you from optimizing modeling details that have no bearing on the decisions your agent ultimately makes.