Pretrained Visual Models and DINO-WM › Summary

• Frozen visual representations as state. A fixed encoder such as DINOv2 supplies per-frame patch features; DINO-WM's transition model also uses short observation and action histories. Patch-level features preserve spatial indexing, but not guaranteed object correspondence.
• The trade. Pretraining can provide useful visual structure without task-specific encoder training, while freezing can hide distinctions needed for control. Test the representation in the intended domain.
• Feature-space dynamics prediction. The dynamics objective predicts next-step frozen features from action-conditioned context. It has no pixel-reconstruction term; an optional diagnostic decoder is separate. Feature MSE remains a proxy for control.
• Normalization. A single global scale changes numeric units without changing Euclidean plan rankings. Per-dimension scaling changes relative weights and can amplify noisy nearly constant dimensions.
• Action-conditioned rollouts. Recursive predictions can drift. Compare one-step, recursive, and fixed-start persistence errors on held-out trajectories, then test horizons in closed loop.
• Planning with goal images. DINO-WM's CEM planner uses terminal feature distance to an encoded goal; the toy uses a summed intermediate cost. A new test goal can be supplied as one image, but dynamics training still requires aligned observation–action trajectories.
• Generalization. Goal-image reuse applies only where the representation and learned dynamics retain the necessary information and action coverage. Scene, camera, modality, and dynamics shifts require empirical checks or adaptation.
• Evaluate closed-loop. Report task success and inspect feature sensitivity to task-critical variables; neither low feature MSE nor a cross-goal scatter plot proves plan ranking within a goal.

The next chapter turns to video generators, which can produce visible future frames rather than only latent predictions. Whether those frames are action-consistent enough to support planning is a separate empirical question.