Recurrent State-Space Models › Worked Example: Inferring a Velocity You Never See

We now build an RSSM for a small controlled linear-Gaussian environment. The experiment exposes true hidden states and the data-generating equations, so we can check filtering, open-loop forecasts, and uncertainty against a known-model reference. It does not illustrate every difficulty of world modeling: in particular, this environment's conditional futures are Gaussian, not multimodal. Training is CPU-sized; runtime depends on hardware and numerical libraries.

The environment is a discrete spring-damper system with position and velocity as its two state variables. Position is observed with sensor noise; velocity is never an input or target during world-model training. Recorded force actions are also supplied. Position, velocity, actions, and process noise jointly determine future motion. With the known model parameters and Gaussian initial distribution, a Kalman filter gives the exact posterior. The position observation map is $C=[1,0]$, and the two-step observability matrix has rows $C$ and $CA=[1,0.1]$, so its determinant is $0.1$: position history contains information about velocity, even though noise prevents exact recovery.

The experiment asks four questions, each mapping to a section of the chapter:

1. Can the model filter? Score posterior decoder means against clean positions, alongside raw-observation echo, a memoryless shrinkage estimate, and a known-model Kalman filter. Beating echo shows lower estimation error, not by itself use of history.
2. Can the model imagine? Observe 10 positions, then predict 40 steps under the recorded future actions. Compare the RSSM ensemble mean with the deterministic baseline and the Kalman conditional mean using the same point-RMSE calculation.
3. How informative are its intervals? Score central sample-quantile intervals against clean future positions, report width and sampling uncertainty, and compare with exact Kalman intervals. These are diagnostics, not a proof of full calibration.
4. What information is recoverable? Fit a linear probe on training representations and test it on held-out sequences. A high velocity $R^2$ supports linear decodability; it does not prove an identifiable physical coordinate, causal use, or Markov sufficiency.

We retain a deterministic GRU with a point decoder as an illustrative baseline. The RSSM has 20,625 parameters and 1,500 updates; the baseline has 10,689 parameters and 800 updates, and their objectives and observation timing differ. This is not a controlled ablation isolating stochastic state. The shared evaluation cutoff, actions, and point-error metric make the resulting forecasts comparable, but cannot attribute their differences to one architectural choice.