Taxonomies of World Models › Worked Example: One Pendulum, Three World Models

Taxonomies become concrete only when you watch the choices play out on real data. So we will spend the rest of the chapter doing exactly that, with a deliberately small system where every quantity can be inspected. Smallness is the point: with sixteen observation dimensions and a two-dimensional true state, we can compute ground truth exactly, plot every trajectory, and attribute every discrepancy to a specific modeling choice rather than to scale, data volume, or infrastructure.

The environment is a damped pendulum with a torque input, simulated with simple Euler integration and a small amount of process noise on the angular velocity. Its true state is two-dimensional: angle $\theta$ and angular velocity $\dot{\theta}$. The physics are elementary:

$$\dot{\theta}_{t+1} = \dot{\theta}_t + \Delta t \, (-\sin\theta_t - 0.15\,\dot{\theta}_t + u_t) + \varepsilon_t, \qquad \theta_{t+1} = \theta_t + \Delta t \, \dot{\theta}_{t+1},$$

where $u_t$ is the applied torque and $\varepsilon_t$ is Gaussian process noise. The agent never sees $(\theta, \dot{\theta})$ directly. Instead it observes a sixteen-dimensional "pixel" vector produced by a fixed nonlinear sensor: a random linear projection of the sufficient features $(\sin\theta, \cos\theta, \dot{\theta})$, passed through a $\tanh$ nonlinearity. The sensor is invertible in principle, so no information is lost, but the observation coordinates are a tangled mixture of the physical state. This setup isolates exactly the phenomenon the state axis is about: the difficulty lies not in missing information but in coordinates, and the experiment will show how much coordinate choice alone is worth.

Into this arena we send three contestants, one from each of the taxonomic positions we just discussed:

• Contestant A, the observation-space linear model, fits linear dynamics directly in the sixteen-dimensional observation space. This is the pixel-space philosophy taken to its cheapest conclusion: no representation learning, just regression where the data lands.
• Contestant B, the learned latent-state model, trains a small encoder to recover the three dynamics-relevant features from nonlinear observations, then advances those features with the known pendulum transition. The supervised state target is a deliberate teaching convenience: it isolates state-estimation error from transition-model error.
• Contestant C, the oracle feature model, receives the true sufficient features $(\sin\theta, \cos\theta, \dot{\theta})$ and applies the same structured transition. No realistic agent receives this state for free, but the contestant provides a noise-limited reference.

We evaluate all three the same way: fit or initialize from thirty training episodes, then roll out sixty steps from ten held-out initial states with held-out action sequences, measuring the root-mean-square error of the predicted angle against ground truth at every horizon. Finally, we estimate process-noise scale from Contestant C's one-step residuals and build a stochastic variant.

Before running the comparison, we expect Contestant A to accumulate structural error because a linear map in observation coordinates cannot express the nonlinear sensor dynamics. Contestant B should approach the oracle when its encoder estimates state accurately, while Contestant C should expose the irreducible error caused by process noise. The comparison is intentionally diagnostic rather than a fair architecture benchmark, because B and C receive a known transition to isolate the cost of state representation.