Vision-Language-Action and World-Action Models › Predictive Models Paired with Action Models

Everything to this point describes a policy: it maps observations and instructions to actions. A world-action system also includes a predictive component, but it need not be a second head on the policy. Prediction may be an auxiliary training target, a separately generated visual plan used by a controller, or a joint action-and-future output. This section examines those couplings and why prediction and control are not the same objective.

There are four distinct motivations, and separating them prevents confusion about what a predictive component is for.

As an auxiliary representation-learning objective. Predicting future states encourages the encoder to retain information about dynamics that pure action imitation might not need. This is the motivation for the predictive objectives in Part VI: Learning World Models at Scale: learning $p(s_{t+1} \mid s_t, a_t)$ rewards features that help predict transitions on the training distribution, while an imitation head can rely on features correlated with an expert's next move. Predictive loss does not guarantee that the model identifies every causal factor, but the distinction matters when deployment shifts the state distribution.

To exploit action-free video. Human videos and teleoperation recordings without reliable action labels are easier to collect than labeled robot trajectories. Observation-only video prediction can learn from those frames directly; an action-conditioned dynamics head needs recorded or inferred actions. VPT illustrates the latter route in Minecraft: an inverse-dynamics model trained on a smaller labeled set assigns pseudo-actions to a large unlabeled video set, which then supervises a behavior-cloned policy. It is an example of obtaining action supervision from video, not of training an action-conditioned dynamics head on frames alone.

To plan. If you can roll the model forward under candidate actions, you can select actions by predicting their consequences rather than merely imitating. For a one-step lookahead with discount factor $0 \leq \gamma < 1$, select $a_t = \arg\max_a \mathbb{E}_{s_{t+1} \sim p(s_{t+1} \mid s_t, a)}[r(s_t, a) + \gamma V^\pi(s_{t+1})]$: the immediate reward is undiscounted, while the estimated continuation value is discounted. Repeating this choice after each observation is a short-horizon version of the model-predictive control loop from Part VII: Planning and Agency.

To verify and to generate data. A sufficiently calibrated model could help screen candidate actions before execution, though a prediction is not a safety guarantee. A generative model can also produce synthetic experience: NVIDIA's DreamGen pipeline generates "neural trajectories" from a video world model and uses them to augment robot-policy training, an instance of the synthetic-data approach in Part VI, Chapter 4: Data Curation and Synthetic Data. Both uses are limited by fidelity, especially away from the training distribution.

The literature contains at least four distinct ways to attach a predictor to an actor, and they fail differently.

Auxiliary head on a shared trunk. One encoder can feed action and prediction heads. GR-1 predicts future video frames and actions through a GPT-style transformer; the prediction loss can shape the shared representation. The objectives may compete for capacity or gradients. Loss weighting is one way to manage that tradeoff; selective gradient flow, separate training schedules, or separate modules are other design choices.

Text-and-image-conditioned video planner with a separate inverse dynamics model. UniPi conditions its video generator on the initial image and a text description of the current goal, then uses an inverse dynamics model to extract actions from the planned frames. The video-generation stage need not take actions as input; the inverse model supplies the embodiment-specific control interface. Its two-stage error chain remains important: the visual plan must be feasible, and the inverse model must turn consecutive planned frames into workable commands.

One transformer for action and future-image generation. WorldVLA trains one autoregressive transformer on action generation and action-conditioned future-image generation. Under an ordinary causal mask, later action tokens in a chunk can attend to earlier predicted action tokens, so an early action error can propagate directly through the chunk. WorldVLA's action-generation mask blocks that access: each chunk action depends on the text and visual inputs rather than preceding generated actions, allowing the chunk actions to be produced in parallel. Its world-model branch still uses causal attention to generate future images conditioned on actions. The failure mode resembles the compounding-error argument in Part IV, Chapter 5: Stochasticity and Uncertainty, but it is action-to-action feedback within a chunk, not an intermediate generated image corrupting the next action.

Latent-action generative models. Instead of predicting in a robot's motor-command space, infer a discrete latent action code from consecutive frames and model the future in that code space. Genie uses this mechanism; the planned Chapter 4 in this part will develop it further. Its learned codes are not tied to named actuator commands, but this does not establish that the same code transfers between robot bodies or can be decoded into their controls.

When one network carries both a policy loss and a prediction loss, their relative weight is a hyperparameter with first-order consequences.

If the predictive term dominates, the trunk may prioritize visual fidelity over control-relevant distinctions. Photorealistic reconstruction does not by itself establish that the representation is sufficient for choosing actions. Some texture or material cues matter for grasping; the concern is spending capacity on details irrelevant to the target task. This motivates latent predictive and energy-based objectives, as discussed in Part V, Chapter 5: Predictive Embeddings, JEPA, and Energy-Based Models.

If the action term dominates, the shared trunk may learn features useful for imitation but weak for prediction. With behavior cloning, both losses can be expectations under a demonstration dataset, yet their numerical scales and gradient magnitudes are not automatically comparable. A cross-entropy action loss and an MSE prediction loss live on different scales, and their ratio can drift over training. On-policy training can also change the state distribution used by the policy objective, shifting the effective balance even if the explicit loss weights stay fixed.

There is no universal resolution, but there are three useful defaults. First, weight the prediction loss by its eventual role: planning requires predictive accuracy over the action candidates and horizons used, whereas an auxiliary prediction head may need less weight if it improves the policy's representation. Tune this weight against separate control and prediction metrics rather than assuming a fixed small value. Second, predict in a learned latent space rather than pixel space when task-irrelevant visual detail would dominate the objective. Third, evaluate the two heads separately. A single aggregate training loss can hide a strong head compensating for a weak one.

This is the most important conceptual point in the chapter, so it is worth stating bluntly: accurate future prediction is neither necessary nor sufficient for good control.

It is not necessary because a reactive policy without an explicit predictor can succeed at many manipulation tasks. It is not sufficient because low average prediction error on observed or demonstrated transitions does not establish accuracy for alternative actions a planner might consider. An action-conditioned model may reproduce likely next frames on the data distribution yet rank counterfactual action rollouts incorrectly, which is the failure that matters when using it to compare actions. A goal-conditioned visual planner needs a different test: whether its proposed future is feasible and its inverse controller can realize it.

One key diagnostic for planning is counterfactual action sensitivity: does the predicted future change in the correct direction, by roughly the correct amount, when the action changes? A model that predicts an accurate future for the demonstrated action but predicts the same future for a counterfactual action with a different true outcome cannot reliably compare those actions, no matter how low its one-step prediction error is. This test helps establish whether an action-conditioned video generator is useful as a world model; action ranking and calibration must also be checked for the intended decision. The distinction is about tested capability, not the name of the architecture. It is related to the criteria in Part I, Chapter 3: The World-Model Qualification Test.

One diagnostic is to take a held-out state and compare model rollouts under a recorded action and a perturbed action. Checking whether the true futures diverge in the same way additionally requires a resettable simulator, repeated controlled trials, or another way to observe both interventions from comparable states; an ordinary logged transition supplies only one outcome. If the model predicts nearly identical futures for actions whose true outcomes differ, its action conditioning may be too weak for planning, even if its average one-step prediction error is low.

A second criterion is value equivalence in the relevant region. If the model is used for planning, the ranking of candidate actions can matter more than pixel-perfect rollouts. A model that underestimates distance traveled might still rank "approach the mug" above "retreat from the mug" in a simple unobstructed task; around obstacles, contact thresholds, or deadlines, the same bias can reverse the preferred plan. This is the insight behind value-equivalent models such as MuZero, which the planned Part VIII chapter addresses. Accuracy and usefulness come apart when the downstream consumer is a decision rather than a reproduction.

One response to the latency tension is to split perception and action generation across two update rates. Scene interpretation may not need to rerun at every motor-control step, but the appropriate rates depend on the task and how quickly the scene changes.

A slower vision-language module can supply embeddings to a higher-rate action module. In NVIDIA's GR00T N1, a vision-language model runs at about 10 Hz and provides intermediate embeddings to a diffusion-transformer action module running at about 120 Hz, trained with flow matching over action chunks. Those embeddings are learned features, not guaranteed explicit "what" and "where" tokens. For the split to work, they must carry enough task-relevant information between updates while the faster module responds to current state inputs.

The decomposition can reduce how often the large vision-language module runs. It also suggests useful debugging questions: did the slower representation miss a scene change, or did the action module respond poorly to a sound representation? Those causes can interact, especially when modules are trained together, so the split does not make failures automatically attributable to one timescale or guarantee easier training.

The cost is that System 1 operates on context that may become stale between System 2 updates. Fresh proprioception and frequent action-chunk replanning can reduce this risk at the fast timescale. Event-triggered System 2 refresh when a visual scene changes is a possible design, not a mechanism established by the GR00T N1 architecture cited here; without such a mechanism, the slower context is refreshed on its scheduled cadence.

Once you have a predictive model paired with a policy, three applications open up.

Imagination. Train policy updates on rollouts inside a learned model, as in Part VIII, Chapter 3: The Dreamer Family. Dreamer still collects new environment experience to update that model; imagined learning does not turn the whole process into an offline computation. A policy optimized against model predictions can exploit favorable model errors. Shorter rollouts, conservative objectives, or model disagreement checks may reduce this risk in some settings; the planned Part VIII chapter on offline and conservative model-based RL will examine these choices.

Data amplification. Rather than optimizing a policy through imagined rewards, generate trajectories with a model and add them to supervised training. This changes the training distribution and therefore the fitted objective; it does not automatically sidestep model exploitation or synthetic-data bias. If a generated trajectory violates a physical constraint and is treated as valid supervision, it can teach the wrong action. Filtering, weighting, and real-world validation are ways to limit that risk.

Evaluation in imagination. Use the model as a cheap approximate simulator to screen policies before spending real robot time. Real-robot evaluation is a major bottleneck in robot-policy research, but an imagined evaluation inherits the model-fidelity limitations described above. Part XI: Evaluation and Understanding treats it as a measurement-instrument problem: the question is whether model errors distort the policy comparisons you are trying to make.

Three failure modes are worth checking where the architecture uses a predictive model for rollout or optimization.

Compounding drift. One-step errors can accumulate when predictions are fed back as inputs, especially when rollouts leave the training distribution. Local sensitivity of the learned dynamics is one contributor, alongside model bias, action choice, and observation noise; no single Lipschitz statistic establishes long-horizon planning quality. Re-anchoring on a reliable observation interrupts the particular chain of feeding predicted state back into the next prediction, but it does not guarantee a global error bound.

Model exploitation. A planner or policy optimizing against an imperfect model can favor actions where its errors are optimistic. This can create a systematic failure that average prediction error hides, although exploitation is not inevitable in every run. Evaluate model quality on the states and actions the optimized policy visits.

Horizon mismatch. The horizon over which the model is accurate can be shorter than the task horizon. A model useful for five-step prediction can still support a fifty-step task through repeated observation and replanning; it cannot justify an uncorrected fifty-step imagined rollout. Recognizing the mismatch tells you whether to replan more often, improve the model, or use a hierarchical decomposition.