Actions, Inverse Dynamics, and Latent Actions › Actions, Inverse Dynamics, and Latent Actions

Every world model has to answer a deceptively simple question: given what the world looks like right now, and something you do next, what happens after? The "something you do" is an action. It is the torque you apply to a joint, the key you press, the token you emit, or the force you apply to a robot arm. In a well-defined simulated environment like a MuJoCo hopper or a game of Doom, that knob is labeled: a three-dimensional torque vector, or one of a dozen key combinations. Learning $p(s_{t+1} \mid s_t, a_t)$ in those settings is a supervised regression problem with clean labels. If you want a bit more intuition, imagine the model as a function that has two knobs to turn: one is fixed by what the world is doing, and the other is fixed by what you are doing. Only the second one is yours to adjust.

Real data is rarely so tidy. Most of the world's video has no action labels. A chef chopping onions, a car merging on the highway, a person opening a door. These are all state transitions, but the underlying actions are invisible. If you want a world model that can be controlled from this kind of data, you first have to infer the missing actions. The labels that would tell you which torque, which key, which token, are simply absent. What remains is a sequence of observations and the transitions they imply.

This chapter is about that gap. It moves in four steps, each with a classical analogue. First, we look at the clean case: what does it mean for a transition to be action-conditioned, and why does the action input deserve a special role compared to other inputs? Second, we look at inverse dynamics, the problem of recovering the action from a state transition, and at controllability, the structural property that determines which transitions are even achievable. Third, we look at how latent actions can be discovered from unlabeled video, where the real actions are never observed. Fourth, we look at how those latent actions can be aligned back to real controls, so a policy learned on top of the world model does something useful when deployed. Each step has a classical analogue in filtering, system identification, planning, and control, and keeping those connections visible helps clarify why the modern formulations look the way they do.

By the end of the chapter you will see that "the action" is not a fixed primitive that a world model inherits from its environment. It is a concept the model has to construct, and the choices you make about how to define it have consequences that ripple through planning, transfer, and controllability. In Part V: World-Model Architectures we saw how different architectures represent states. Here we ask what gets to sit in the action slot of those architectures.

It is worth saying up front that "model quality" is not one thing. A world model can be good at predicting future observations while still failing as a controllable object. It can represent the state well for downstream classification while representing the action poorly for planning. It can produce calibrated uncertainty in one region of state space and miscalibrated certainty in another. And it can be decision-useful for a particular task while being spectacularly wrong about the world in ways the task never probes. As we go through this chapter, try to keep these four notions of quality separate: predictive fidelity, representation quality, uncertainty quality, and decision usefulness. Most of the interesting failures in the rest of the book come from confusing one of these for another.