Predictive and Generative Objectives › Key Parameters
Michael BrenndoerferJuly 1, 20261 min read
World Models HandbookAll chapters Part 6 of 12Learning World Models at Scale
Predictive and Generative Objectives › Key ParametersOne section · Sections in this chapter9 / 12 ↓
Sections in this chapter12
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The key parameters for this implementation are:
- hidden: Number of hidden units in the MLP layers (32). Larger values increase capacity but also training time and overfitting risk.
- lr: Learning rate for the Adam optimizer (1e-2). Controls the step size for both the one-step and multi-step objectives.
- K: Rollout horizon for the multi-step training loss (5). Longer horizons expose the objective to more recursively generated states but cost more compute and memory.
- batch_size: Number of trajectories sampled per multi-step update (32). Larger batches reduce gradient variance but use more memory.
- num_iterations: Number of gradient updates for each objective (2000). More iterations can improve fit but also increase the chance of overfitting to the training trajectories.
- T: Trajectory length used both for data generation and evaluation rollouts (20). It sets the maximum horizon over which held-out rollout error is measured.