Frontier Questions and Research Roadmaps › Frontier Questions and Research Roadmaps

Consider a hypothetical demonstration: someone hands you a model and a folder of results. The model takes a short history of camera frames and a sequence of joint commands, and predicts the next frame. It was trained on tens of thousands of hours of video and robot interaction data, it renders hand deceleration, cloth, and liquid plausibly, and its average pixel error on held-out clips is the best anyone has recorded. The folder shows the model's outputs to a viewer, and the viewer cannot reliably tell them from the real footage. Everyone in the room agrees the model is excellent.

Then someone asks a decision-relevant question for the robot: if I move my gripper 3 cm to the left, does the cup end up in it?

The reported evaluation does not establish an answer. Average error under the tested action distribution does not establish the consequence of this particular intervention, especially if it lies outside that distribution. The model might answer correctly, but the supplied evidence does not verify that capability. This distinction between a capability and evidence for it is the subject of this final chapter.

Average pixel error measures a chosen image-space discrepancy, not necessarily perceptual realism or distributional fidelity. It can detect physical errors visible in evaluated frames. But averaging can dilute errors in rare, decision-relevant situations. It supplies no distribution-free accuracy guarantee for actions outside the evaluation population. An action-effect test addresses that gap by measuring outcomes under the queried actions rather than inferring them from an aggregate visual score.

Throughout this handbook we separated four things that are easy to conflate: a model's predictions, the representation those predictions are computed in, the decisions an agent makes using them, and the evidence that any of it works on a new system. Part I introduced the world-model idea and the qualification test. Part II gave us state, belief, identification, and control. Part III, Part IV, Part V, Part VI, Part VII, Part VIII, and Part IX built up representations, architectures, learned dynamics, and foundation-scale systems. Part X and Part XI examined applications and how to evaluate them. Part XII, this part, has been about reliability: failure modes, calibration and safe control, security and governance, systems, and in Sim-to-Real Deployment and Continual Correction the machinery for moving a model from a simulator into a physical system and keeping it current.

This chapter is the closing one. It does not introduce a new architecture or a new benchmark. It converts everything before it into frontier questions you can pursue, plus three small executable experiments that make the central distinctions concrete. There is no next chapter; the roadmap here is meant to be handed to you. The experiments are deliberately small enough to run on a laptop, and that is the point: the distinctions they expose are conceptual, so they do not need scale to become visible. What they need is care in construction, and that care is what the rest of the chapter is about.

The roadmap turns broad aspirations into specified questions, measurements and comparisons. A precise protocol does not guarantee an easy answer: representation learning, optimization, identification and implementation can remain difficult. It does make clear which claim a result would address.

Three questions organize the chapter:

1. What would it mean for a world model to be general-purpose rather than embodiment-specific?
2. What do causal, compositional, and persistent structure require that passive predictive training does not provide?
3. What must evaluation measure if prediction quality is not the same as decision usefulness?

Each question is hard on its own, and each is easy to answer vacuously. "The model generalizes across embodiments" is vacuous unless you name the source and target systems, the held-out population, the adaptation budget, and the criterion. "The model has causal structure" is vacuous unless you name the intervention and the assumption that makes the conclusion valid. "The model has memory" is vacuous until you say what information is retained, for how long, how it is refreshed, and what happens when it goes stale. Precision here is not pedantry; it is the difference between a research program and a slogan.

The three questions share a practical requirement: specify what evidence would support the claim. Generality needs meaningful query interfaces and competence on a declared population. Evidence for learned structure needs suitable experiments, representations and identification assumptions. Evaluation needs a population, a protocol and decision-relevant measurements; weighting is one part. Model capacity and training data can affect all three, but do not replace these specifications.