Actions, Inverse Dynamics, and Latent Actions › Summary

The action slot of a world model is where control enters. Unlike an ordinary state feature, it is the interface through which a planner selects among modeled futures.

We covered four ideas in this chapter.

• Action-conditioned transitions. The forward model $p(s_{t+1} \mid s_t, a_t)$ is where actions live. They enter via concatenation, FiLM conditioning, cross-attention, or as event inputs, and the choice affects what the model tends to learn. A model that ignores its actions can still predict, but it cannot plan.
• Inverse dynamics and controllability. The inverse model $p(a_t \mid s_t, s_{t+1})$ is a mirror of the forward model. It is used for goal-conditioned control, auxiliary representation learning, pseudo-labeling, and hierarchical control. Controllability is the structural property of the transition pair that says which states are reachable with actions; learned representations require separate one-step diagnostics and rollout tests.
• Learning actions from unlabeled video. When actions are missing, you can train an inverse model on a small labeled set and apply it to the rest, or learn a latent variable directly from transitions. Bottlenecks and regularizers constrain the solutions but do not guarantee physical action semantics.
• Latent action alignment with real controls. Learned latent actions are only defined up to an invertible reparametrization. Aligning them with real actions takes a small amount of supervision, applied as a linear probe, a decoder head, a contrastive objective, or a discrete codebook assignment. The amount of alignment needed depends on what you plan to do downstream.

The next chapter, Interaction Data and Passive Observation, turns to the data side of this story. Where does the transition data come from? What does it look like when it was produced by an agent actively trying to learn versus by a passive observer, and what can and cannot be inferred from each? The methods in this chapter are the tools you use once you know what data you have. The next one is about looking at the data itself.