Predictive Embeddings, JEPA, and Energy-Based Models › Key Parameters
Michael BrenndoerferJune 29, 20261 min read
World Models HandbookAll chapters Part 5 of 12World-Model Architectures
Predictive Embeddings, JEPA, and Energy-Based Models › Key ParametersOne section · Sections in this chapter7 / 10 ↓
Sections in this chapter10
Reading tools Definitions & inline links
Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.
Article links
Make inline references clickable
The worked example exposes several parameters that control the predictive embedding:
- D_LATENT: Dimension of the learned embedding. It limits coordinate capacity and should be selected with task-level evaluation. A smaller latent can strengthen a capacity bottleneck, but it does not by itself determine whether nuisance detail or decision-relevant state is discarded.
- ema: Exponential moving average decay for the target encoder. Smaller values track faster and may reduce stability in recipes that rely on a slow target. The appropriate schedule depends on the update timescale; this toy uses 0.99.
- var_weight: Weight on the VICReg-style variance hinge. It sets the strength of the direct anti-collapse pressure.
- epochs: Number of optimization steps. Too few leaves the predictor undertrained; too many can expose slow collapse.
- lr: Adam learning rate for the online encoder and predictor.
- Predictor hidden size: Capacity of the predictor MLP. In the simplified analysis, a sufficiently capable predictor can represent part of the context-to-target regression; practical capacity must be validated with the rest of the recipe.