Tokenized Game and Environment Models › Summary

Tokenized game and environment models encode observations as sequences of discrete symbols. An observation tokenizer $E$ maps frames to $z_t \in \{1,\ldots,K\}^L$; one possible action-conditioned model autoregressively predicts the next tokens, reward, and termination. Tokenization can shorten the sequence relative to a specified raw-pixel representation, while the cost of sampling and planning still depends on $K$, $L$, and the dynamics architecture.

The chapter's key points are summarized below.

• Tokenization is a representation choice. The tokenizer $E$ is not a neutral preprocessing step. It decides which features the dynamics model can reason about, and it can have low reconstruction error while discarding the variable that determines whether the environment is solvable.
• The autoregressive factorization is a design template. A discrete token model with an action-conditioned prior, plus reward and termination heads, is one convenient structure. Alternatives exist and are legitimate, and different choices trade capacity, training stability, and sampling speed.
• Time alignment is part of the model. Action repeat, no-op controls, stacking, and control latency affect the transition data. If timing information needed for prediction is absent from the model inputs, increasing model size alone cannot supply it.
• Interactive generation is not the same as decision usefulness. A model can be a great playable environment and a bad planner, or vice versa. Three uses demand three validation targets: playability, planning value, and demonstration quality.
• Separate branch tests from closed-loop tests. One-step accuracy does not certify a world model. Fixed-action branches test multi-step transition fidelity; adaptive policy runs in the real environment test control usefulness.
• Data support and out-of-distribution queries matter. A neural token model may emit a prediction even on poorly supported inputs. Coverage, calibration, uncertainty checks, or conservative planning can help determine whether a proposed plan should be trusted.
• Transfer has separable components. Visual codes, dynamics priors, action semantics, and decision heads can succeed or fail independently. Cross-game results should specify which mechanism is tested, how the model is adapted, and what baselines are used.

The next chapter takes the representation question one step further. Instead of training a discrete tokenizer and an autoregressive dynamics model over its codes, we will look at models that keep a continuous feature space from a frozen pretrained visual backbone and predict in that space. The same core question applies: does the learned transition preserve the distinctions the agent needs, or does it only preserve the ones that make the video look right?