Mathematical and Dynamical Foundations › Mathematical and Dynamical Foundations

In Chapter 2 we described a world model informally: an encoder that turns observations into a state, a dynamics model that predicts how that state evolves, and a way of reading rewards or outcomes back out. Every one of those pieces rests on a small body of mathematics that this chapter puts in place. You do not need years of control theory to follow the rest of this handbook, but you do need four ideas to be solid: how we represent uncertainty with random variables and conditional distributions, how states evolve through transitions to form trajectories, how those trajectories settle onto fixed points and attractors, and how information theory tells us which parts of the past a state actually needs to carry.

It is worth pausing on why precisely these four. The first idea supplies the language: without conditional distributions you cannot even state what a world model claims about the future. The second supplies the object: trajectories built from local transition laws are what a world model produces and what training data consist of. The third supplies the diagnostic: long-horizon behavior is governed by stability and attractors, and ignoring them is the single most common way a technically competent model fails in deployment. The fourth supplies the design principle: a finite state cannot hold an infinite past, so every practical world model is an exercise in principled forgetting, and information theory is what turns "forgetting" from a euphemism into a measurable quantity.

We will lean on one running example throughout: a thermostat controlling the temperature of a room. It is almost embarrassingly simple, yet it contains a stochastic transition law, a limit-cycle attractor, a hidden state variable that temperature alone cannot capture, and a sharp lesson about the difference between predicting well one step ahead and reproducing the behavior of a system over long horizons. By the end of the chapter you will have simulated it, fit models to it, and watched a model with excellent one-step accuracy fail in closed loop for reasons that are purely dynamical. The thermostat earns its place precisely because nothing about it is exotic: if these failure modes appear in a system with one hidden bit, they will appear with interest in robotics, finance, and language.

If you have seen probability and linear algebra before, treat this chapter as a calibration pass: the goal is not novelty but shared vocabulary, stated assumptions, and a few intuitions, especially about stability and compounding error, that most machine-learning curricula skip entirely. Readers encountering the material for the first time should work through the code cells rather than skimming them; each one is short, and each one demonstrates a claim that would otherwise remain an assertion.