Frontier Questions and Research Roadmaps › What Generality Would Actually Require

For this chapter, we separate two useful senses of "general"; they are an organizing convention rather than an exhaustive taxonomy.

The first sense is distributional generality: demonstrated performance over a specified range of inputs from a stated family. A broad video corpus or many manipulation trajectories can support training for that goal, but dataset size alone does not establish it. Report performance on held-out populations and any subpopulations for which the broad average hides poor results.

The second sense is functional generality: competence at answering the same kind of question across declared domains, with shared query and output semantics. A shared model may accept a new robot through an explicit embodiment map without retraining its shared dynamics. Whether that works depends on the target range, available embodiment information and evaluation results, not just a common output format.

A system can have the first without the second. A generator with no action-query interface does not directly provide the action-conditioned queries used here, even if its videos score well. A generator that does expose such an interface needs a separate assessment of its action semantics and transfer performance. Neither visual plausibility nor an interface alone establishes a universal world model; any demonstrated generality remains relative to the systems, tasks and budgets actually evaluated.

Broader data can improve both coverage and learned action semantics; interface redesign can change which data is usable. Neither intervention guarantees either form of generality. Evidence for one does not automatically establish the other, although a designed transfer experiment can support both. Bounded cross-embodiment world-model results already exist: He et al. (2026 revision) use particle states and displacement actions, map native joint commands through forward kinematics, and deploy the same model for manipulation with distinct robotic hands. This is evidence within their evaluated manipulation setting, not arbitrary-system universality.

For the planning interface used here, an agent needs to query candidate actions from an initialized state or belief. A predictor without any action-query mechanism does not directly supply that interface. Initialization might use an observation encoder, measured state, a supplied initial state or another declared procedure; a learned encoder is not mandatory. Action conditioning makes alternative-action prediction possible, but does not by itself identify a causal intervention or an individual counterfactual. Those interpretations need assumptions and evidence. Reward and policy heads are optional components, not substitutes for those conditions.

The opening scenario conflates four capabilities that can share a name. Separate them when interpreting a result: evidence for one need not establish another.

• Predictive video generation. In this setting, produce future observations conditioned on past observations. Fidelity, temporal coherence, and physical plausibility are possible evaluation criteria (Part IX, Chapter 3). Video generation also includes unconditional and text-conditioned tasks; none inherently requires an action interface. Ho et al., "Video Diffusion Models" study these distinct settings.
• Action-conditioned dynamics prediction. Predict the next state given the current state and an action. This supplies an interface for an alternative-action query: "what if I applied this torque instead?" Interpreting its prediction causally also requires appropriate coverage and identification assumptions; an action input alone does not supply them (Part VI, Chapters 2 and 3).
• A sufficient, actionable state representation. A representation that retains the information needed for the specified decisions. In a partially observed problem, the reference is the available observation–action history or its belief, not necessarily one observation. Any approximate sufficiency claim needs a task, policy class and tolerance. Part III, Chapter 2 developed the criteria; Part VIII, Chapter 5 showed the decision-centric version in practice.
• Closed-loop competence. The whole system, including planner or policy, achieves the objective in the real environment under noise, delay, and perturbation. This is the only capability that directly corresponds to "the robot picks up the cup."

These sit in a dependency chain, but not a strict one. Action conditioning without a good state can fail. A good state without calibrated dynamics can still support a robust policy if the planner is conservative. Closed-loop competence can sometimes be achieved without any of the others, using a reactive policy that never plans. Part IX, Chapter 6 examined vision-language-action models that take exactly that route.

The practical consequence: when someone claims a new system is "a world model," ask which of the four capabilities were measured, on what population, and with what action interface.

A compact way to keep the four capabilities separate is to maintain an evidence checklist. The following matrix is an illustrative prioritization, not a measured score or a logical implication. Its values mean primary (1), supplementary (0.5), or not selected (0) in this example. Calibration alone establishes neither state sufficiency nor closed-loop competence. Test sufficiency with task-relevant information and control comparisons, and test competence by executing the system under the stated conditions.

If only generative fidelity was measured, that evidence alone does not establish control competence. Keep the four capabilities as separate columns in your notes, and record which measurements support each. This lets you assess a paper's particular claims without assuming that evidence for one capability proves another.

For cross-embodiment claims, a transfer contract with five clauses makes the tested shift and available information explicit:

1. Source and target systems. Name them. A source of "many robot arms, mostly parallel-jaw, tabletop, 6-DoF" and a target of "a 7-DoF arm with a soft gripper and a mobile base" is a specific, falsifiable statement. "Robots" is not.
2. Observation and action spaces. What are the dimensions, units, frames, and semantics before and after transfer? Does the target have the same action units (joint position deltas? end-effector velocities? torques?) or does the map require a learned or analytic adapter?
3. Held-out population. Which tasks, objects, embodiments and scenes were excluded from training, including pretraining? Distinguish held-out instances from unseen categories, and document what the corpus audit can and cannot verify (Part VI, Chapter 4).
4. Training information and adaptation budget. Zero-shot, few-shot with $k$ demonstrations, or fine-tuned with $n$ gradient steps and $m$ environment interactions. An adaptation report without a budget does not fully specify the transfer comparison.
5. Evaluation criterion. Task success rate? Return? Sample efficiency relative to a specialist baseline? Robustness to perturbation? Each answers a different question (Part XI, Chapters 3 and 4).

Population, adaptation budget and criterion jointly define the transfer question. A broad held-out population with little adaptation tests a different claim from specialization on a narrow population with a generous budget. These settings cannot be ordered by difficulty from breadth and budget alone: the target tasks, available information and success criteria may differ. Two papers can both report "transfer" while answering different questions. State these conditions before comparing their results.

The Open X-Embodiment effort provides a multi-robot dataset and RT-X policy models. Its reported positive transfer concerns policies trained with experience from multiple platforms under the paper's evaluation protocol. That supports a defined cross-robot policy-transfer result, not a claim that an action-conditioned world-dynamics model transfers universally. A dynamics-transfer study needs its own prediction and control tests even if it uses the same dataset.

Contrast the well-specified version with the aspirational version: "a single model that can be deployed on any embodied agent." That claim has no held-out population, no adaptation budget, and no criterion, so it does not define a reproducible empirical test or a clear refutation criterion. It is a research direction, and it should be labeled as one. The difference between a research direction and a research result is not rhetorical. A direction tells you where to spend effort; a result tells you what is currently true. Confusing the two can lead teams to build on unverified capability claims and reviewers to accept claims without explicit evaluation or refutation criteria.

For the cross-embodiment planning system proposed here, what interfaces would make its claims testable?

Consider a system with observation $o_t \in \mathbb{R}^{d_o}$, latent or environment state $s_t \in \mathbb{R}^{d_s}$, belief $b_t$ over $s_t$ (Part II, Chapter 2), action $a_t \in \mathcal{A}$, transition law $p(s_{t+1} \mid s_t, a_t)$, reward $r_t \in \mathbb{R}$, and policy $\pi(a_t \mid b_t)$. One useful system-level interface contract includes:

• An encoder from heterogeneous observations $o_t$ to a state or belief. It must handle sensor geometry that differs between platforms: different camera intrinsics and extrinsics, different numbers of views, different proprioceptive channels.
• A transition law conditioned on actions with explicit embodiment-specific meanings and units. A canonical frame with adapters is one option; embodiment-conditioned models or separate native interfaces are others. Joint torques and base velocities require a declared interpretation rather than an assumed correspondence. Representation, training coverage and scale can all affect transfer.
• A constraint interface. Joint limits, self-collision, contact, and actuator saturation differ per embodiment. A separate checker can supply constraints instead of embedding them in the learned dynamics. Optimistic impossible predictions can mislead a planner if its objective selects those regions; exploitation is a risk, not an inevitable outcome (Part XII, Chapter 1).
• A notion of what is decision-relevant. Different tasks on identical dynamics can require different task-relevant compressions. A full Markov state, or a sufficiently rich shared representation, can still be sufficient for both (Part III, Chapter 2).

Sufficiency must be assessed relative to the declared family of tasks. A model that serves pick-and-place tasks need not serve locomotion tasks on the same robot: a compression optimized for one task may discard information needed by another. A task change can therefore require a richer or different compression, but it does not necessarily do so. The same full Markov state can support several rewards or goals; training and representation choices determine how much of that information a learned model retains.

Two intermediate designs are worth testing under a declared transfer contract:

• Broad pretraining plus adapters. Train one large dynamics model on heterogeneous data, then attach small per-embodiment modules (action encoders, constraint heads, residual correctors) that are fit with a small budget. This keeps the expensive shared part and localizes the embodiment-specific part. The research question is how much of the shared part transfers across embodiments.
• Specialist models with classical identification. Fit each platform with a structured model (Part II, Chapter 3). A justified model structure can reduce the information needed to estimate its parameters, but any data-efficiency claim needs a named comparator, matched target accuracy and measured data budgets. Include this specialist as an informative target-specific baseline rather than assuming which approach will win.

The adapter approach tests how much of a shared model remains useful after embodiment-specific adaptation. Compare adapters with the unadapted model, and report what changes under the declared budget. The specialist supplies a separate target-specific comparison, not a universal lower bound. Failing to beat it does not show that pretraining supplied no transfer: a pretrained model can improve over the same architecture trained from scratch while still losing to the specialist. Estimate that transfer benefit with matched initialization comparisons or target-data learning curves, alongside the specialist result.

Transfer is not free. Negative transfer occurs when training on the source distribution makes the target worse than training on the target alone. It shows up in practice for a few identifiable reasons:

• Action-space mismatch. Ambiguous or lossy action mappings can bias action-effect prediction on affected embodiments. Aggregate visual metrics can hide that bias; a compromise does not necessarily bias every embodiment.
• Conflicting dynamics. Without information identifying the domain, identical inputs with opposite action effects are ambiguous. A squared-loss point predictor averages the conditional outcomes, which may be inappropriate for either domain. Context conditioning or a distributional predictor can represent the ambiguity differently.
• Capacity or objective conflict. With a restricted model and training budget, fitting other domains can conflict with target performance. Shared structure can also benefit the target; capacity is not necessarily partitioned into a fixed share per domain. Experiment 1 studies a different issue, compression of nuisance versus task-relevant coordinates, not cross-domain negative transfer.
• Calibration collapse. A model that was well calibrated on the source can be badly miscalibrated on the target. Prediction accuracy and calibration can both affect safety; their relevance depends on the declared safety objective and how the controller uses predictions and uncertainty (Part XII, Chapter 2).

The countermeasure is the same as the diagnosis: make the action interface semantically explicit, evaluate action-effect prediction per embodiment rather than in aggregate, and always report the specialist baseline on the target population. Each of the four causes above suggests a specific diagnostic:

• Action-space mismatch: compare predicted and realized action effects per embodiment.
• Conflicting dynamics: inspect residuals by domain and action. Bimodality can be informative when components are sufficiently separated, but its absence does not exclude conflicting dynamics.
• Capacity dilution: test whether a performance gap shrinks under controlled enlargement. This is one diagnostic, not identification of capacity dilution; optimization or representation changes can also shrink the gap.
• Calibration collapse: inspect a calibration plot on the target distribution.

These diagnostics make specific transfer failures measurable. None uniquely identifies a cause: enlarged residuals, for example, can reflect several changes at once. Their cost depends on access to representative target data and interventions.