Autoregressive Token World Models › Autoregressive Token World Models

Imagine supplying a game model with an initial agent position shifted by seven cells modulo sixteen. Does the model return to the observed trajectory, follow a different but valid trajectory, or make further mistakes? We will test those possibilities in a small deterministic world. The preceding chapters introduced recurrent state and transformer memory; here we focus on a particular interface for dynamics: discrete observation and action tokens predicted by a causal transformer.

A world model predicts environment-relevant futures from available observations and actions. A recurrent model carries a learned summary; a token model can retain an explicit recent token record and compute predictions from it. Both use conditional sequence factorizations, but lossy tokenization changes the predicted outcome space: a distribution over token grids is not identical to a distribution over raw images. Causal-transformer optimization and attention machinery are reusable, while modality alignment, tokenization, loss masks, and evaluation still need design. Whether scaling behavior transfers is an empirical question.

IRIS demonstrates categorical next-token world modeling for Atari control. Related interactive generators use different decoders: Genie generates discrete video frames with iterative masked-token prediction, while Oasis generates frames autoregressively using latent diffusion. Temporal autoregression, discrete representation, and within-frame next-token decoding are separate choices. We will use IRIS's general categorical sequence pattern for the miniature and distinguish the alternatives when they arise.

The pipeline has three stages, and each gets its own treatment in this chapter:

• Tokenization: observations and actions are mapped to symbols. A fully unified design combines disjoint modality vocabularies; hybrid systems may retain continuous inputs or separate prediction heads.
• Next-token dynamics: a causal transformer learns the conditional distribution over future world tokens given the past token stream, including action tokens that we inject along the way.
• Rollout and use: the model generates hypothetical futures by feeding its own predictions back as inputs, and an agent (in Part VII) queries those futures to make decisions.

The apparent simplicity hides design decisions. A tokenizer can discard distinctions needed by a task, although history or other modalities may retain information absent from one frame. Serialization and masks determine which inputs a prediction may read; chronological ordering alone does not establish causal identification. Under teacher forcing, training uses recorded histories. Imperfect autoregressive models may generate different histories at deployment, and their errors can influence later predictions. We will measure rollout errors and corrective fine-tuning without treating every mismatch or declining exact-match curve as proof of exposure bias.

The runnable miniature keeps the original three-entity environment, two training budgets, self-generated-context fine-tuning, and a one-token intervention. Matched teacher-forced metrics, an additional-training control, and an exact environment predictor help separate what these experiments do and do not establish.