Games and Learned Game Engines › Summary

Atari, board games, and open-ended interactive environments stress different properties: hidden state in pixel observations, branching search under known rules, and composition or persistence over longer tasks.

• Atari challenges models to retain action-relevant state from high-dimensional, partially observable observations.
• Board games distinguish AlphaZero's search with known rules from MuZero's learned, decision-oriented dynamics.
• Long-horizon open-world tasks often stress compositionality and memory, but the requirements depend on the task.

Latent dynamics are central to PlaNet, Dreamer, and MuZero; visual generation is central to DIAMOND, GameNGen, and Genie. World Models combined visual compression with latent prediction. The design trade-offs include decision targets, renderability, action conditioning, context, and measured compute rather than a universal latent-versus-pixel speed ranking.

Longer recursive use requires measuring both prediction error and specified rule behavior. Under the displayed same-action and Lipschitz assumptions, the error bound is linear for $L=1$ and may grow exponentially for $L>1$; it does not fit a particular rollout. Conditioning augmentation, temporal context, memory, and re-anchoring are candidates for testing, not guaranteed fixes.

An imperfect model can still support useful decisions. Dreamer-style imagination updates and model-based search are distinct uses, and both can be misled by reward or transition errors. The grid-world planner succeeds on its seeded test set even though its observation predictions drift; that does not establish transfer beyond this example.

The next chapter turns to robotic manipulation, where contact, sensing, and data collection change what a useful model must predict. We should carry the evaluation habit forward, not assume that a game result transfers unchanged.