Mathematical and Dynamical Foundations › Summary

• Random variables and conditional distributions are the substrate of world modeling. A world model is a stack of conditionals, and the chain rule composes them into full trajectory distributions. Conditioning both shifts and sharpens belief; models that report only conditional means discard the spread that planning needs.
• States, transitions, and trajectories formalize prediction. The transition kernel $p(s_{t+1} \mid s_t, a_t)$ generalizes the deterministic map, and the Markov property, an assumption about the state representation, licenses recurrences instead of ever-growing histories. Rollout uncertainty compounds, saturating for stable systems and exploding for unstable ones.
• Fixed points, stability, and attractors characterize long-run behavior. Linearization classifies fixed points via Jacobian eigenvalues, and attractors range from fixed points through limit cycles to chaos. Judge models of chaotic systems on statistical fidelity, not long-horizon point accuracy.
• Information, compression, and sufficient statistics say what a state must remember. The data processing inequality bounds what encoding can preserve, and a Markov state is exactly a sufficient statistic of the past for predicting the future, the ideal that state abstraction and latent representations approximate.
• The thermostat tied it all together: a limit-cycle attractor, a hidden one-bit state that temperature alone misses, and a vivid demonstration that one-step accuracy does not imply correct closed-loop dynamics.

With these foundations in place, the next chapter can do what this one deliberately deferred: attach actions, rewards, and optimality to the transition structure, arriving at the Markov decision process, the formal arena in which nearly all of Part II takes place.