Vision-Language-Action and World-Action Models › Limitations and Impact

One durable limitation of current VLA and world-action models is the cost and coverage of embodied data. Teleoperation requires an operator, robot, workspace, and resets; the resulting trajectories are not interchangeable with language-model text tokens, so comparing their raw counts gives no meaningful scale factor. Open X-Embodiment assembled more than a million trajectories across many contributors, yet each target robot and task may still have sparse coverage. Language backbones contribute visual and semantic priors, predictive objectives can use observation-only video when designed for it, and cross-embodiment pooling may help in compatible settings. None of these removes the need to measure control performance on the target distribution.

The second limitation is evaluation. Real-robot trials are slow and variable across setups. Action prediction error is a tempting substitute because it is cheap and automatic, but the code above shows that it measures a different quantity from closed-loop success. Matched simulation and distributed pairwise testing can broaden evaluation, while still needing real-robot validation and explicit uncertainty. Benchmark contamination is a further concern: repeated tuning on a task suite weakens later held-out claims. The chapter's checklist is a partial defense, not a guarantee.

Safety is the third limitation. A trained VLA's offline loss alone gives no general guarantee of safe behavior on a physical system that can break objects, damage itself, or injure a person. Limited guarantees may come from a separately verified controller, monitor, or constrained operating envelope; they do not follow from average action-prediction accuracy. A predictive head might help screen actions, but its errors need testing in the off-distribution situations where safety matters. Optimization against an imperfect predictor can also favor its blind spots. Runtime monitoring, conservative action limits, and human fallback are measures to consider, not features that every research VLA deployment already has. Part XII: Reliable World Models treats their systematic development as an open problem.

The gains are conditional but useful. A language-conditioned interface can reduce the need to redesign task-specific symbols or code for some supported instructions; a new sentence alone does not guarantee a new skill. Cross-embodiment training can reuse experience when observation and action conventions are compatible. Open X-Embodiment released data from 22 robot embodiments, while the reported RT-X training experiments used nine and showed both gains and losses across target evaluations. Predictive models add imagined rollouts and auxiliary signals to existing options such as offline demonstrations, simulation, and model-based learning; they did not invent policy learning without physical execution. The next part of this book asks how these approaches hold up as tasks grow longer and failures become more costly.