TD-MPC and Control-Centric Representations › Summary

Michael BrenndoerferJuly 17, 20262 min read
World Models HandbookAll chapters
Part 8 of 12Decision-Centric Research Lineages
Chapter 47TD-MPC and Control-Centric Representations › Summary
Reading section 8 of 9 · Summary · 2 minSection 8 of 9 · 2 min

Book contents

World Models Handbook12 parts · 72 chaptersView the whole book →
Active chapter 47 of 72

Section 8 of 9 · 2 min read

TD-MPC and Control-Centric Representations › SummaryOne section · Sections in this chapter8 / 9 ↓
Sections in this chapter9
Reading tools Definitions & inline links

Choose your expertise level to adjust how many terms are explained. Beginners see more tooltips, experts see fewer to maintain reading flow. Hover over underlined terms for instant definitions.

Article links

Make inline references clickable

  • TD-MPC blends three traditions. Sampling-based model predictive control searches action sequences, temporal-difference learning trains the bootstrapped tail value, and control-centric representation learning trains TD-MPC's decoder-free TOLD latent state.
  • Five learned components, two objectives. A temporally weighted model/critic objective combines reward, value, and latent-consistency losses; the policy has a separate action-value objective. Model-unrolled latent states receive training gradients.
  • No decoder, deliberately. Reconstruction fidelity is not control sufficiency. Removing its loss can free optimization pressure for task-related predictions, but does not guarantee better features and may remove a helpful auxiliary signal; usefulness depends on the task and data.
  • Planning is short but not free. The original MPPI-style controller scores sampled sequences, weights elites with a max-shifted softmax, and refits a Gaussian. It does not z-score returns. Planning cost scales with samples, iterations, and horizon.
  • The terminal value supplies the tail. The released planner appends γHmin⁡jQθ,j(z^t+H,a~)\gamma^H\min_j Q_{\theta,j}(\hat z_{t+H},\tilde a) for a sampled terminal policy action a~\tilde a. The analytic LQR example instead uses the oracle V∗=Q∗(⋅,a∗)V^*=Q^*(\cdot,a^*); a learned critic and policy can help or hurt depending on their errors at candidate terminal states.
  • Training and planning both use the model. Training unrolls predicted latents along replayed actions, while the released TD target bootstraps from encoded observed successors. Planning searches fresh candidate actions in the learned model.
  • TD-MPC2 changes the implementation materially. It adds SimNorm and widespread LayerNorm, distributional reward/value heads, and all-component task conditioning. Reported image runs use a shallow CNN, not DINOv2; the 104-task online suite and 80-task offline generalist are distinct.
  • Failure modes remain. Search can exploit learned-score errors, and deterministic dynamics cannot explicitly represent outcome distributions. The short toy run is a diagnostic rather than evidence that these issues are solved.