Games and Learned Game Engines › Learned Simulators and Playable Generation

A learned simulator replaces some or all hand-coded transitions with a model trained on interaction data. A frame predictor outputs a next image from recent images and actions; an interactive generator must also keep responding over many steps. Both are world models, but the right architecture and tests depend on whether the output is used for planning, training, or play.

The purpose matters, because it changes which errors are consequential. Two uses are useful to distinguish.

• Simulation for agents. The model generates transitions used for planning or imagined policy updates. Reward, value, termination, and action consequences must be accurate enough for the chosen decision process; photorealistic reconstruction is optional. Dreamer and MuZero are examples with learned latent dynamics.
• Playable generation for humans. The system renders an action-responsive view under a latency budget. Visual continuity, controllability, and rule behavior matter together. GameNGen and the Genie systems are examples, but their tasks and hardware differ.

The representation follows that objective. A compact latent can reduce the cost of simulating candidate futures, but its efficiency depends on model size, rollout length, hardware, and any decoder or search. A renderable system may still predict compressed visual tokens rather than raw pixels. Ha and Schmidhuber's World Models is already a hybrid: it learns visual compression and latent dynamics, then reconstructs frames for inspection. The practical comparison is task- and implementation-specific, not an overnight-versus-month rule.

Building on Recurrent State-Space Models, the latent lineage encodes observations and predicts future latent states under actions. PlaNet combines a deterministic recurrent path with stochastic latent variables. Dreamer uses imagined latent trajectories to improve its actor and critic, while collecting additional environment interactions to train and update the world model. DreamerV2 and V3 extended that line. A stochastic latent can represent multiple possible futures, though deterministic models remain useful when their assumptions fit the task. With $r_t$ denoting reward after action $a_t$ and $c_t$ denoting whether the episode continues after that action, one generic transition description is

$$\begin{aligned} z_{t+1} &\sim p(z_{t+1} \mid z_t, a_t), \\ (r_t,c_t) &\sim p(r_t,c_t \mid z_t,a_t,z_{t+1}), \end{aligned}$$

where:

• $z_t$: a latent representation at time $t$ that may summarize observation history
• $a_t$: the action taken at time $t$
• $p(z_{t+1} \mid z_t, a_t)$: the learned transition distribution over the next latent given the current latent and action
• $r_t$: the reward after action $a_t$, whose value the model predicts during imagination
• $c_t$: the continuation indicator after action $a_t$ (one if the episode continues, zero if it terminates)
• $p(r_t,c_t \mid z_t,a_t,z_{t+1})$: a generic reward-and-continuation prediction; individual implementations may condition these heads on only a subset of the listed variables

The sampled transition can represent more than one possible next latent for the same inputs. Whether the latent should be interpreted as a belief over hidden states depends on the inference model and observation history; stochasticity alone does not establish that interpretation. The indexing above keeps the after-action reward aligned with the worked example below.

MuZero trains representation, dynamics, and prediction networks for search targets without an observation-reconstruction objective. Its learned latent need not depict the board or Atari frame. That can avoid spending model capacity on visual details irrelevant to its search targets; it does not prove a general compute advantage over every reconstructive model. Search cost still depends on model architecture and the number of nodes expanded.

The relevant intuition is decision sufficiency: a model need not reconstruct everything to support a particular search objective. If the task only requires choosing an action in two visually different states, preserving the action ranking can suffice. Equal action-conditional rewards and values are a stronger sufficient condition, not a necessary one. A new task or action set may require distinctions the representation discarded.

Ha and Schmidhuber's World Models (2018) combined a VAE for image compression, an MDN-RNN for latent dynamics, and a small controller. The VizDoom controller was trained in a generated dream environment; the CarRacing experiment used a different training arrangement. The result is an influential early hybrid of learned compression and dynamics.

The visual-generation projects differ in both training data and output. DIAMOND (2024) used a diffusion world model to train an Atari agent and separately demonstrated an interactive model trained on static Counter-Strike: Global Offensive gameplay. GameNGen (2024) first recorded DOOM play from an RL agent, then trained an action-conditioned diffusion frame predictor; its reported interactive rate was about 20 frames per second on a single TPU. Genie (2024) learned latent actions from unlabeled video for controllable generation. That training description should not be copied verbatim onto every successor. Genie 2 generated action-controlled 3D scenes for up to a minute in reported examples, most of which lasted 10–20 seconds. More recently, Genie 3 reported real-time 720p interaction at 24 frames per second with consistency over a few minutes. These are reported demonstrations under different settings, not matched benchmark scores.

The differences between these methods come down to a small set of choices.

• Observation representation. Pixels, latent vectors, discrete tokens. Each has different compression efficiency and different reconstruction quality.
• Dynamics model. Recurrent latent prediction (Dreamer), token prediction (for example, IRIS), or diffusion-based visual prediction (DIAMOND and GameNGen).
• Conditioning data. Recorded actions are available for GameNGen's gameplay data; Genie learns a latent action representation from video without action labels.
• Temporal context. A longer input history can help with hidden state or visual continuity, but increases memory and compute in architecture-dependent ways.

These choices interact, but none fixes long-horizon behavior automatically. The right design depends on whether the output is used for decisions, rendered to a player, or evaluated as a research model.

Playable generation places stricter human-facing requirements on three properties that can also matter to an agent-facing simulator.

• Controllability. Actions must causally change the generated world, with feedback fast enough for the intended interaction. A game that ignores a button press is not playable, regardless of visual quality.
• Interactive throughput. Latency must be low enough for the intended control loop. GameNGen reports about 20 frames per second on a single TPU; that is a measured configuration, not a universal threshold.
• Persistence. Revisiting a location should reveal a coherent continuation of the earlier scene when the task requires it. A fixed context window may make that difficult, but its limit must be measured for the particular model.

One-step frame quality alone does not establish playability. A practical evaluation should record the model, hardware, frame rate and input-to-output latency; test whether changing an action changes the next outcome; compare multi-step trajectories and revisited locations against a reference when one exists; count specified rule violations; and, if agent training is the goal, evaluate the resulting policy in the reference environment. Short attractive clips and downstream return answer different questions. None of these measurements by itself certifies a complete game engine.

We will return to memory and consistency in the next section. For now, the takeaway is that learned simulators are not one thing. They are a family of compromises between fidelity, speed, controllability, and abstraction, and the right point on that surface depends on whether the consumer is an agent computing gradients or a human holding a controller. Choosing the wrong point can make an otherwise excellent model useless for your purpose, which is why the "what is it for" question has to be answered first.