Frontier Questions and Research Roadmaps

Consider a hypothetical demonstration: someone hands you a model and a folder of results. The model takes a short history of camera frames and a sequence of joint commands, and predicts the next frame. It was trained on tens of thousands of hours of video and robot interaction data, it renders hand deceleration, cloth, and liquid plausibly, and its average pixel error on held-out clips is the best anyone has recorded. The folder shows the model's outputs to a viewer, and the viewer cannot reliably tell them from the real footage. Everyone in the room agrees the model is excellent.

Then someone asks a decision-relevant question for the robot: if I move my gripper 3 cm to the left, does the cup end up in it?

The reported evaluation does not establish an answer. Average error under the tested action distribution does not establish the consequence of this particular intervention, especially if it lies outside that distribution. The model might answer correctly, but the supplied evidence does not verify that capability. This distinction between a capability and evidence for it is the subject of this final chapter.

Average pixel error measures a chosen image-space discrepancy, not necessarily perceptual realism or distributional fidelity. It can detect physical errors visible in evaluated frames. But averaging can dilute errors in rare, decision-relevant situations. It supplies no distribution-free accuracy guarantee for actions outside the evaluation population. An action-effect test addresses that gap by measuring outcomes under the queried actions rather than inferring them from an aggregate visual score.

Throughout this handbook we separated four things that are easy to conflate: a model's predictions, the representation those predictions are computed in, the decisions an agent makes using them, and the evidence that any of it works on a new system. Part I introduced the world-model idea and the qualification test. Part II gave us state, belief, identification, and control. Part III, Part IV, Part V, Part VI, Part VII, Part VIII, and Part IX built up representations, architectures, learned dynamics, and foundation-scale systems. Part X and Part XI examined applications and how to evaluate them. Part XII, this part, has been about reliability: failure modes, calibration and safe control, security and governance, systems, and in Sim-to-Real Deployment and Continual Correction the machinery for moving a model from a simulator into a physical system and keeping it current.

This chapter is the closing one. It does not introduce a new architecture or a new benchmark. It converts everything before it into frontier questions you can pursue, plus three small executable experiments that make the central distinctions concrete. There is no next chapter; the roadmap here is meant to be handed to you. The experiments are deliberately small enough to run on a laptop, and that is the point: the distinctions they expose are conceptual, so they do not need scale to become visible. What they need is care in construction, and that care is what the rest of the chapter is about.

The roadmap turns broad aspirations into specified questions, measurements and comparisons. A precise protocol does not guarantee an easy answer: representation learning, optimization, identification and implementation can remain difficult. It does make clear which claim a result would address.

Three questions organize the chapter:

1. What would it mean for a world model to be general-purpose rather than embodiment-specific?
2. What do causal, compositional, and persistent structure require that passive predictive training does not provide?
3. What must evaluation measure if prediction quality is not the same as decision usefulness?

Each question is hard on its own, and each is easy to answer vacuously. "The model generalizes across embodiments" is vacuous unless you name the source and target systems, the held-out population, the adaptation budget, and the criterion. "The model has causal structure" is vacuous unless you name the intervention and the assumption that makes the conclusion valid. "The model has memory" is vacuous until you say what information is retained, for how long, how it is refreshed, and what happens when it goes stale. Precision here is not pedantry; it is the difference between a research program and a slogan.

The three questions share a practical requirement: specify what evidence would support the claim. Generality needs meaningful query interfaces and competence on a declared population. Evidence for learned structure needs suitable experiments, representations and identification assumptions. Evaluation needs a population, a protocol and decision-relevant measurements; weighting is one part. Model capacity and training data can affect all three, but do not replace these specifications.

For this chapter, we separate two useful senses of "general"; they are an organizing convention rather than an exhaustive taxonomy.

The first sense is distributional generality: demonstrated performance over a specified range of inputs from a stated family. A broad video corpus or many manipulation trajectories can support training for that goal, but dataset size alone does not establish it. Report performance on held-out populations and any subpopulations for which the broad average hides poor results.

The second sense is functional generality: competence at answering the same kind of question across declared domains, with shared query and output semantics. A shared model may accept a new robot through an explicit embodiment map without retraining its shared dynamics. Whether that works depends on the target range, available embodiment information and evaluation results, not just a common output format.

A system can have the first without the second. A generator with no action-query interface does not directly provide the action-conditioned queries used here, even if its videos score well. A generator that does expose such an interface needs a separate assessment of its action semantics and transfer performance. Neither visual plausibility nor an interface alone establishes a universal world model; any demonstrated generality remains relative to the systems, tasks and budgets actually evaluated.

Broader data can improve both coverage and learned action semantics; interface redesign can change which data is usable. Neither intervention guarantees either form of generality. Evidence for one does not automatically establish the other, although a designed transfer experiment can support both. Bounded cross-embodiment world-model results already exist: He et al. (2026 revision) use particle states and displacement actions, map native joint commands through forward kinematics, and deploy the same model for manipulation with distinct robotic hands. This is evidence within their evaluated manipulation setting, not arbitrary-system universality.

For the planning interface used here, an agent needs to query candidate actions from an initialized state or belief. A predictor without any action-query mechanism does not directly supply that interface. Initialization might use an observation encoder, measured state, a supplied initial state or another declared procedure; a learned encoder is not mandatory. Action conditioning makes alternative-action prediction possible, but does not by itself identify a causal intervention or an individual counterfactual. Those interpretations need assumptions and evidence. Reward and policy heads are optional components, not substitutes for those conditions.

The opening scenario conflates four capabilities that can share a name. Separate them when interpreting a result: evidence for one need not establish another.

• Predictive video generation. In this setting, produce future observations conditioned on past observations. Fidelity, temporal coherence, and physical plausibility are possible evaluation criteria (Part IX, Chapter 3). Video generation also includes unconditional and text-conditioned tasks; none inherently requires an action interface. Ho et al., "Video Diffusion Models" study these distinct settings.
• Action-conditioned dynamics prediction. Predict the next state given the current state and an action. This supplies an interface for an alternative-action query: "what if I applied this torque instead?" Interpreting its prediction causally also requires appropriate coverage and identification assumptions; an action input alone does not supply them (Part VI, Chapters 2 and 3).
• A sufficient, actionable state representation. A representation that retains the information needed for the specified decisions. In a partially observed problem, the reference is the available observation–action history or its belief, not necessarily one observation. Any approximate sufficiency claim needs a task, policy class and tolerance. Part III, Chapter 2 developed the criteria; Part VIII, Chapter 5 showed the decision-centric version in practice.
• Closed-loop competence. The whole system, including planner or policy, achieves the objective in the real environment under noise, delay, and perturbation. This is the only capability that directly corresponds to "the robot picks up the cup."

These sit in a dependency chain, but not a strict one. Action conditioning without a good state can fail. A good state without calibrated dynamics can still support a robust policy if the planner is conservative. Closed-loop competence can sometimes be achieved without any of the others, using a reactive policy that never plans. Part IX, Chapter 6 examined vision-language-action models that take exactly that route.

The practical consequence: when someone claims a new system is "a world model," ask which of the four capabilities were measured, on what population, and with what action interface.

A compact way to keep the four capabilities separate is to maintain an evidence checklist. The following matrix is an illustrative prioritization, not a measured score or a logical implication. Its values mean primary (1), supplementary (0.5), or not selected (0) in this example. Calibration alone establishes neither state sufficiency nor closed-loop competence. Test sufficiency with task-relevant information and control comparisons, and test competence by executing the system under the stated conditions.

If only generative fidelity was measured, that evidence alone does not establish control competence. Keep the four capabilities as separate columns in your notes, and record which measurements support each. This lets you assess a paper's particular claims without assuming that evidence for one capability proves another.

For cross-embodiment claims, a transfer contract with five clauses makes the tested shift and available information explicit:

1. Source and target systems. Name them. A source of "many robot arms, mostly parallel-jaw, tabletop, 6-DoF" and a target of "a 7-DoF arm with a soft gripper and a mobile base" is a specific, falsifiable statement. "Robots" is not.
2. Observation and action spaces. What are the dimensions, units, frames, and semantics before and after transfer? Does the target have the same action units (joint position deltas? end-effector velocities? torques?) or does the map require a learned or analytic adapter?
3. Held-out population. Which tasks, objects, embodiments and scenes were excluded from training, including pretraining? Distinguish held-out instances from unseen categories, and document what the corpus audit can and cannot verify (Part VI, Chapter 4).
4. Training information and adaptation budget. Zero-shot, few-shot with $k$ demonstrations, or fine-tuned with $n$ gradient steps and $m$ environment interactions. An adaptation report without a budget does not fully specify the transfer comparison.
5. Evaluation criterion. Task success rate? Return? Sample efficiency relative to a specialist baseline? Robustness to perturbation? Each answers a different question (Part XI, Chapters 3 and 4).

Population, adaptation budget and criterion jointly define the transfer question. A broad held-out population with little adaptation tests a different claim from specialization on a narrow population with a generous budget. These settings cannot be ordered by difficulty from breadth and budget alone: the target tasks, available information and success criteria may differ. Two papers can both report "transfer" while answering different questions. State these conditions before comparing their results.

The Open X-Embodiment effort provides a multi-robot dataset and RT-X policy models. Its reported positive transfer concerns policies trained with experience from multiple platforms under the paper's evaluation protocol. That supports a defined cross-robot policy-transfer result, not a claim that an action-conditioned world-dynamics model transfers universally. A dynamics-transfer study needs its own prediction and control tests even if it uses the same dataset.

Contrast the well-specified version with the aspirational version: "a single model that can be deployed on any embodied agent." That claim has no held-out population, no adaptation budget, and no criterion, so it does not define a reproducible empirical test or a clear refutation criterion. It is a research direction, and it should be labeled as one. The difference between a research direction and a research result is not rhetorical. A direction tells you where to spend effort; a result tells you what is currently true. Confusing the two can lead teams to build on unverified capability claims and reviewers to accept claims without explicit evaluation or refutation criteria.

For the cross-embodiment planning system proposed here, what interfaces would make its claims testable?

Consider a system with observation $o_t \in \mathbb{R}^{d_o}$, latent or environment state $s_t \in \mathbb{R}^{d_s}$, belief $b_t$ over $s_t$ (Part II, Chapter 2), action $a_t \in \mathcal{A}$, transition law $p(s_{t+1} \mid s_t, a_t)$, reward $r_t \in \mathbb{R}$, and policy $\pi(a_t \mid b_t)$. One useful system-level interface contract includes:

• An encoder from heterogeneous observations $o_t$ to a state or belief. It must handle sensor geometry that differs between platforms: different camera intrinsics and extrinsics, different numbers of views, different proprioceptive channels.
• A transition law conditioned on actions with explicit embodiment-specific meanings and units. A canonical frame with adapters is one option; embodiment-conditioned models or separate native interfaces are others. Joint torques and base velocities require a declared interpretation rather than an assumed correspondence. Representation, training coverage and scale can all affect transfer.
• A constraint interface. Joint limits, self-collision, contact, and actuator saturation differ per embodiment. A separate checker can supply constraints instead of embedding them in the learned dynamics. Optimistic impossible predictions can mislead a planner if its objective selects those regions; exploitation is a risk, not an inevitable outcome (Part XII, Chapter 1).
• A notion of what is decision-relevant. Different tasks on identical dynamics can require different task-relevant compressions. A full Markov state, or a sufficiently rich shared representation, can still be sufficient for both (Part III, Chapter 2).

Sufficiency must be assessed relative to the declared family of tasks. A model that serves pick-and-place tasks need not serve locomotion tasks on the same robot: a compression optimized for one task may discard information needed by another. A task change can therefore require a richer or different compression, but it does not necessarily do so. The same full Markov state can support several rewards or goals; training and representation choices determine how much of that information a learned model retains.

Two intermediate designs are worth testing under a declared transfer contract:

• Broad pretraining plus adapters. Train one large dynamics model on heterogeneous data, then attach small per-embodiment modules (action encoders, constraint heads, residual correctors) that are fit with a small budget. This keeps the expensive shared part and localizes the embodiment-specific part. The research question is how much of the shared part transfers across embodiments.
• Specialist models with classical identification. Fit each platform with a structured model (Part II, Chapter 3). A justified model structure can reduce the information needed to estimate its parameters, but any data-efficiency claim needs a named comparator, matched target accuracy and measured data budgets. Include this specialist as an informative target-specific baseline rather than assuming which approach will win.

The adapter approach tests how much of a shared model remains useful after embodiment-specific adaptation. Compare adapters with the unadapted model, and report what changes under the declared budget. The specialist supplies a separate target-specific comparison, not a universal lower bound. Failing to beat it does not show that pretraining supplied no transfer: a pretrained model can improve over the same architecture trained from scratch while still losing to the specialist. Estimate that transfer benefit with matched initialization comparisons or target-data learning curves, alongside the specialist result.

Transfer is not free. Negative transfer occurs when training on the source distribution makes the target worse than training on the target alone. It shows up in practice for a few identifiable reasons:

• Action-space mismatch. Ambiguous or lossy action mappings can bias action-effect prediction on affected embodiments. Aggregate visual metrics can hide that bias; a compromise does not necessarily bias every embodiment.
• Conflicting dynamics. Without information identifying the domain, identical inputs with opposite action effects are ambiguous. A squared-loss point predictor averages the conditional outcomes, which may be inappropriate for either domain. Context conditioning or a distributional predictor can represent the ambiguity differently.
• Capacity or objective conflict. With a restricted model and training budget, fitting other domains can conflict with target performance. Shared structure can also benefit the target; capacity is not necessarily partitioned into a fixed share per domain. Experiment 1 studies a different issue, compression of nuisance versus task-relevant coordinates, not cross-domain negative transfer.
• Calibration collapse. A model that was well calibrated on the source can be badly miscalibrated on the target. Prediction accuracy and calibration can both affect safety; their relevance depends on the declared safety objective and how the controller uses predictions and uncertainty (Part XII, Chapter 2).

The countermeasure is the same as the diagnosis: make the action interface semantically explicit, evaluate action-effect prediction per embodiment rather than in aggregate, and always report the specialist baseline on the target population. Each of the four causes above suggests a specific diagnostic:

• Action-space mismatch: compare predicted and realized action effects per embodiment.
• Conflicting dynamics: inspect residuals by domain and action. Bimodality can be informative when components are sufficiently separated, but its absence does not exclude conflicting dynamics.
• Capacity dilution: test whether a performance gap shrinks under controlled enlargement. This is one diagnostic, not identification of capacity dilution; optimization or representation changes can also shrink the gap.
• Calibration collapse: inspect a calibration plot on the target distribution.

These diagnostics make specific transfer failures measurable. None uniquely identifies a cause: enlarged residuals, for example, can reflect several changes at once. Their cost depends on access to representative target data and interventions.

The second frontier question concerns structure: what kind of structure can learned world models acquire, and what does acquiring it require?

"Structure" is not a single property that a model either has or lacks. Temporal structure, causal structure, compositional structure, and persistent-memory structure are different properties with different requirements. A model can have one and lack another. Evidence for one property alone need not establish the others, although a well-designed test may support more than one. The next three subsections separate these questions.

An observational conditional describes the sampled data-generating process. Without additional assumptions, it need not identify an intervention distribution. More observations from that same process do not resolve an ambiguity between observationally equivalent causal models. Known structure, suitable adjustment variables, randomized actions or other identification assumptions can change what the data establish.

Three distinct notions get merged in casual discussion, and Pearl's causal ladder gives the standard separation:

• Association. "Given that the agent has been moving right, what comes next?" Temporal prediction is one example of an associational query. This is the lowest level in the causal-information hierarchy, not necessarily the easiest prediction problem computationally.
• Interventional prediction. "If I force the agent to move left, what happens?" Identifying this effect requires appropriate assumptions: randomized actions, valid adjustment for confounding, or other justified structure. An explicit causal diagram alone need not identify the effect.
• Counterfactual inference. "Given what actually happened, what would have happened had I acted differently?" This also requires a justified account of the individual circumstances or latent disturbances shared across the factual and counterfactual scenarios. Observational fit alone does not generally identify it.

Imagine a manipulation dataset collected by an expert that only pushes toward the goal. Accuracy on that distribution does not validate predictions for pushing away from it. Those predictions can be wrong or miscalibrated; confidence and error are not inevitable, since structural assumptions may support extrapolation. Coverage, architecture, uncertainty estimation and evaluation all influence the outcome. An independent intervention test makes this unsupported action query explicit (Part XII, Chapter 1).

Training loss on expert trajectories need not reveal errors at actions those trajectories never cover. A model can fit the observed distribution while extrapolating poorly elsewhere. A deterministic planner that treats its predictions as exact can then exploit those errors. Broader action coverage, calibrated uncertainty, conservative planning and appropriate structural assumptions are possible remedies. None guarantees correctness outside the conditions under which it was validated; changing the architecture alone does not supply missing evidence.

Causal representation learning attempts to recover variables that behave causally, so that interventions and counterfactuals become well defined. The program is laid out in Schölkopf et al., "Towards Causal Representation Learning", and interventional approaches such as Ahuja et al. show identifiability results under explicit assumptions.

State the assumptions of the particular result. Ahuja et al.'s Theorem 5.3 uses a polynomial decoder with full-column-rank coefficients, support regularity, a hard intervention fixing one latent coordinate, and nonempty interior for the remaining intervention support. The learned decoder has the prescribed polynomial degree, the encoder does not collapse the support, reconstruction is exact on observational and interventional supports, and one encoded coordinate is fixed on the intervention support. Under these conditions, that intervened latent is identified up to shift and nonzero scaling. Their block-affine result uses different intervention-support and representation constraints. These are conditional coordinate-identification results, not proof that arbitrary robot latents recover complete causal dynamics. Check the particular theorem rather than applying one generic checklist to every approach.

The blunt consequence: a learned disentangled representation is not causal evidence. If you train a $\beta$-VAE and the latent dimensions align with your factors, or if an object-centric slot attention module produces one slot per object, you have found a representation with a certain factorization. You have not shown that intervening on the object corresponding to slot 3 will change the state in the way your model predicts, because you have not run the intervention. The claim and the evidence are different objects.

Separating a representation into components does not establish how interventions work. Even a probability factorization need not imply independence: the chain rule $p(x,y)=p(x)p(y\mid x)$ also holds for dependent variables. An independent product makes an additional statistical claim, but neither separation nor independence alone identifies the effect of forcing a component to change. A representation can align with observed factors while leaving their intervention behavior unidentified. The assumptions of a particular causal-identification result, rather than the factorization alone, establish what intervention claims it supports.

The next cell constructs a linear confounded system. The associational predictor sees only $X$. An information-advantaged structural reference also observes the common cause $Z$. This is not identification with an unobserved confounder. We report two distinct estimands: noisy individual-outcome error on held-out observational samples, and error in the interventional mean averaged over $Z$ at three fixed intervention values. Their absolute magnitudes are not directly comparable.

Planners select actions, and their model queries can include actions absent from the recorded data. Test the claimed action effects with an appropriate intervention protocol: apply specified actions and compare repeated outcomes with the corresponding predicted quantities. A discrepancy challenges the prediction or its assumptions; it does not identify whether the cause is a wrong causal coefficient, unmodeled context, sampling error or another failure. Agreement on the tested interventions supplies scoped evidence, not proof of every structural claim.

Compositional generalization is the ability to handle novel combinations of factors that were each observed separately. The clean experimental design is a held-out-combination split: every factor level appears in training, but selected combinations do not.

Sequence-to-sequence studies found substantial generalization gaps under compositional evaluations (Keysers et al.; Hupkes et al.; Lake and Baroni). Their protocols differ: compound-divergence splits, excluded function-pair combinations, and primitive or length splits are not identical tests. An analogous factor-combination split can be designed for dynamics learning: cover each factor level in training, hold out selected friction–payload or shape–surface pairs, and evaluate damping, contact or action-effect predictions separately on the held-out cells. This is a proposed dynamics evaluation, not a dynamics-transfer result from those sequence studies.

Two questions keep the interpretation of this split clear:

1. Are separate factor effects identifiable under the stated model and assumptions? If payload co-varies perfectly with friction, their separate effects need not be identifiable without additional assumptions. For a specified linear-in-parameters family, full column rank establishes coefficient uniqueness and conditioning diagnoses numerical or noise sensitivity; neither establishes identifiability for every nonlinear model. This question matters when interpreting recovered factor effects, but separate-effect identification is not a prerequisite for every held-out-combination performance test: a known structural law can supply those effects without estimating them from the training split.
2. The evaluation must distinguish seen from unseen combinations. Report both separately. A single aggregate over seen and unseen cells can conceal their contrast, even when it shows some overall performance change.

The interesting failure mode is when an additive inductive bias is wrong. If the true dynamics multiply two context factors, an additive model can fit the training combinations well and be systematically wrong off the diagonal. The second experiment below constructs exactly this case, and then deliberately breaks the multiplicative assumption too, so that the interaction model's success is not unconditional either.

A held-out-combination split probes the interaction of data coverage, model capacity, and inductive bias. A misspecified restricted family can retain error even with more samples, whereas a richer family may represent the interaction. Rank checks establish identifiability only within the chosen feature family. Additional structural assumptions can resolve ambiguities that the observed combinations alone do not resolve, but those assumptions must be stated and tested.

"Memory" in world models covers at least four mechanically distinct mechanisms. Conflating them makes it impossible to say what an ablation measured.

• Recurrent hidden state. The internal state of an RNN or a recurrent state-space model (Part V, Chapter 1). It carries a compressed summary of history, with no guaranteed storage policy and no explicit retrieval.
• A belief state. The sufficient statistic of the history for the current latent state under the model's assumptions (Part II, Chapter 2). A belief is a filtered, uncertainty-aware summary. Under particular models a compact parametric belief can be much smaller than the stored history; a general continuous-state posterior need not have a fixed-size representation.
• External memory. This includes distinct mechanisms, not just one episodic database. Graves et al. introduced Neural Turing Machines with differentiable external memory and a learned read/write controller. Blundell et al. used non-parametric episodic state-action return memory for control. Lewis et al. provide a retrieval-augmented generation example in NLP, not a control result or an origin claim for all retrieval augmentation. A key-value or vector store can retain experience without updating model weights, but retrieval errors, staleness and storage costs still need evaluation.
• Causal finite-window transformer context. Past tokens inside a configured window are attended over directly (Part V, Chapter 2). This is one implementation, not a definition of all transformer attention. Access does not guarantee reliable retrieval. Tokens outside the window are not directly attended to unless another mechanism, such as a summary, recurrent state, or external store, retains their information.

Each mechanism has possible failure modes. A learned recurrent state can drift and can be difficult to inspect; that difficulty depends on the representation. Beliefs can be systematically miscalibrated if the model is wrong. External stores can retrieve the wrong entry, or the right entry from the wrong episode. Dropping a token removes its direct attention path; the effect on a decision depends on redundancy and any retained summaries, so behavioral forgetting need not be a hard discontinuity.

Four questions should accompany any memory claim:

1. What information is retained? Episode identity, hidden context, task goal, object permanence, a recently observed cue.
2. How is it refreshed? Never? On a schedule? When prediction error rises? Is there a hazard rate for context change? The third experiment shows that a memory without a refresh mechanism is worse than no memory at all once the context changes.
3. How much does it cost? Storage, retrieval latency, and the compute and energy per decision (Part XII, Chapter 4).
4. What are the reset semantics? When does memory clear? At episode boundaries, at task changes, at a detected distribution shift? Retention across a reset can carry stale evidence when the reset changes that information's validity. Other retained facts, such as unchanged robot geometry, may remain useful.

The four mechanisms differ not just in what they retain but in how they fail to be updated:

• A recurrent state is updated at each invoked recurrent step, which need not be every environment step. Its learned update rule may be poorly matched to the rate at which the world changes.
• A belief is updated by a filter whose assumptions may be wrong.
• A standard rolling FIFO context window drops tokens as new ones arrive, without a relevance judgment. Selective retention policies can use relevance instead.
• An external store changes through its configured write, eviction, and retention rules; an agent need not control every update. Stored information can outlast the context in which it was valid.

Each mechanism needs an explicit retention and refresh policy. The third experiment isolates fixed and updating beliefs rather than comparing these storage architectures.

More retained information need not improve the decisions of a fixed learned agent. The following possible failures show why retention and use must be evaluated together.

• Stale context. The goal or physics changed and memory did not. Part VI, Chapter 5 and Part XII, Chapter 5 discussed continual correction in this setting.
• Retrieval of wrong evidence. A nearest-neighbor retrieval returns a transition from a visually similar but dynamically different situation.
• Distractor amplification. Memory preserves salient but irrelevant evidence, and a learned policy trained to attend to memory can over-weight it.
• Compression loss. Summarizing history discards exactly the rare detail that made the current situation special.

For this toy, memory is an estimate of a task-relevant belief. Other memory systems may retain raw history rather than estimate a sufficient statistic. Evaluate the retained information and update mechanism under the context changes they will actually face. For an ablation of history access, match current, nonhistorical inputs and training and compute budgets as far as the design permits. Historical information is the intended treatment, not an input confound to remove. Extra parameters or different training can change the comparison; document such differences and use additional controls before attributing the effect to history access alone.

A lossy history summary can discard information; lossless coding or an invertible representation need not do so. If the discarded information is irrelevant to the declared task, a summary can reduce storage while preserving what that task needs, but computing and maintaining it also costs resources. If an omitted detail matters under a later condition, the summary may no longer be sufficient. Report performance across the expected conditions and their frequencies, alongside retention and update costs, rather than relying on an average alone.

The third frontier question is about measurement. Auditing whether current metrics support the intended decision claim is a practical research priority proposed here; its payoff depends on the task and available evaluation data.

Part XI introduced five complementary evaluation protocols: teacher-forced one-step prediction, recursive open-loop rollout, intervention-response comparison, closed-loop planning, and transfer. These are not strictly nested tests. Results under one protocol do not by themselves establish performance under another with different inputs or objectives.

These metrics measure different quantities, not progressively more precise versions of one quantity. An image-space loss measures its chosen discrepancies; pixel MSE alone does not establish perceptual fidelity. A calibration metric measures correspondence between stated probabilities or intervals and realized events. A closed-loop metric measures performance on a declared goal. A link between these quantities needs its own assumptions or evidence; a score label does not supply that link.

Consider this conceptual progression of quantities. It is not a universal ordering of evaluation cost or decision relevance:

• One-step prediction loss. Mean squared error or negative log-likelihood of $o_{t+1}$ given the history and action. With densely recorded targets, scoring can be inexpensive relative to new data collection. Its weighting depends on the chosen loss and units; raw observation MSE can be dominated by large actual residuals, not necessarily by the highest-variance coordinates.
• Recursive rollout error. Error after $H$ autoregressive steps with the model consuming its own predictions (Part XII, Chapter 1 analyzed compounding).
• Calibration. Whether stated probabilities or prediction intervals agree with realized events. Calibration is one input to a safety argument, not validation of the complete safety filter; the decision rule and operating assumptions also need testing (Part XII, Chapter 2).
• Interventional prediction error. Error in predicted outcomes under specified interventions, which may be inside or outside the logging policy's support. Such tests directly probe action effects. Other validated prediction and rollout tests can also inform planning; an intervention mean is not an individual counterfactual.
• Action ranking accuracy. Given a state and a set of candidate action sequences, does the model order them the way the environment does? Ranking matters for an argmin over that set. Numerical value scales also matter when a planner applies thresholds, risk penalties or trade-offs between objectives.
• Closed-loop return and success. The decision-relevant outcome, measured in the real or high-fidelity environment, preferably with the evaluator blinded to the model identity where feasible.
• Safety-relevant outcomes. Constraint violations, minimum separation, peak force, recovery latency.
• Latency and adaptation cost. How long a decision takes, and what recovery after a shift requires under a declared criterion. Samples, compute, wall time and energy are possible adaptation-cost measures.

One-step observation loss alone generally does not establish the later outcomes in this list. The next experiment shows why for a particular task. Raw mean squared error weights coordinates equally in the chosen units. High-variance coordinates can dominate when their actual squared prediction errors are large; variance is not an explicit metric weight. A decision depends on the coordinates the task needs, which need not have high variance. More data can reduce prediction errors, even to zero in an appropriate setting, but does not redefine the metric's weighting or supply a universal implication from aggregate error to decision performance. A restricted model family may admit such a link; state and test its assumptions.

Evaluation can depend on the model without being circular. Distinguish population selection from using the model's own predictions as ground truth:

• A policy trained in the model can be evaluated validly in an independent simulator or physical system. The evaluation becomes self-scoring if the learned model also supplies the outcome used as truth.
• Resemblance between logging and evaluation policies does not establish their action-support overlap. Report which states and actions were covered, and which target queries require extrapolation; different policies can also share support.
• If the metric is a learned critic, then a metric improvement can reflect critic overfitting rather than model improvement.

Use an independent outcome evaluator, audit the target state-action coverage, and report prediction metrics alongside realized closed-loop outcomes when assessing a controller. A different collection policy is one possible coverage intervention, not a guarantee of new support. The first experiment uses a task-like evaluation policy distinct from the exploratory training policy. Obtaining additional simulator or hardware evidence has a cost that must be measured for the application.

Model-generated trajectories need not stay in already fitted regions. Scoring them against independent outcomes can reveal errors; reusing the same model as the truth source can instead hide them. Keep tuning instances separate from reporting instances, and document how the evaluation policy selects its population. A fixed independent simulator or physical measurement supplies an outcome check that the learned model does not define.

Two statistical checks matter for interpreting a reported result: overlap between sampling units and the random sources varied by repeated runs.

Leakage across related transitions. Random transition splits can place strongly related samples from one rollout on both sides of the split, making the score optimistic for generalization to new episodes. Correlation alone does not prove memorization, and adjacent samples need not be near-duplicates. Use trajectory-level or episode-level splits when new-episode performance is the question, and task-level splits for new-task performance. Grouping episodes removes within-episode overlap, but shared environments, duplicated trajectories and preprocessing fitted on the test set still require separate checks.

Seeds are not automatically independent systems. Report what each seed varies: initialization, data selection, environment trajectories, task draws or combinations of these. Repeated runs on a fixed physical configuration estimate variability under the random sources actually changed; they do not by themselves sample new arms, camera mounts or floors. If the deployment claim spans those configurations, include representative configurations as sampling units. When only a few were tested, state that limitation without treating seed count as configuration count.

Keep the sampling unit tied to the claim. An error bar over initialization-only repeats answers a different question from one over changed hardware configurations. If evaluation trajectories also vary, the repeat variability includes that randomness. Report the protocol, tested configurations and conditional uncertainty rather than labeling all seed variance as either training noise or deployment risk. There is no general numerical lower-bound relation between these different quantities.

With those principles in hand, we can run three small experiments that make the abstract points concrete.

An aggregate observation metric can rank two models in the opposite order from their usefulness for a particular control task. We construct an example, not a general ranking of world-model architectures. All learned models receive the same transitions and actions. The state-focused model additionally receives the declared task-relevant coordinate set, so this is an information-abstraction comparison rather than a parameter-matched architecture benchmark.

The observation contains a small controlled state and a large, persistent background. A model can devote its limited latent capacity to reconstructing that background, reduce observation error, and still omit the coordinates the controller needs. We measure the variance hierarchy, per-coordinate prediction errors, and real-simulator control costs rather than assuming that the intended mechanism occurred.

The decision-relevant state is $s_t=(x_t,v_t)\in\mathbb R^2$. Both coordinates and the scalar action use dimensionless units. The environment evolves as

$$\begin{aligned} x_{t+1} &= 0.85x_t+0.2v_t,\\ v_{t+1} &= 0.8v_t+0.2a_t-0.01v_t|v_t|,\qquad a_t\in[-1,1]. \end{aligned}$$

Here $x_t$ is the positioning coordinate, $v_t$ is a motion coordinate, and $a_t$ is a normalized command. This is a discrete synthetic law, not an Euler approximation to a specified physical robot. The decay terms prevent the unbounded random walk that a long randomly accelerated free particle would exhibit. Starting from $|x_0|\leq1.5$ and $|v_0|\leq1$, the rectangle $|x|\leq1.5$, $|v|\leq1$ is invariant: the first update has magnitude at most $0.85(1.5)+0.2=1.475$, and the monotone velocity map on $[-1,1]$ has magnitude at most $0.8+0.2-0.01=0.99$. The small drag term makes the learned linear predictors approximations rather than exact copies of the plant.

An autonomous nuisance state $n_t\in\mathbb R^2$ follows

$$n_{t+1}=0.95n_t+\sqrt{1-0.95^2}\,\epsilon_t,\qquad \epsilon_t\sim\mathcal N(0,I_2).$$

The initial $n_0\sim\mathcal N(0,I_2)$ makes this process stationary in distribution, with unit marginal variance. Its eight observed channels are $d_t=20n_tW$, where $W\in\mathbb R^{2\times8}$ has its first row equal to one in the first four columns and zero elsewhere, and its second row equal to one in the final four columns. Each nuisance channel therefore has marginal standard deviation 20, but the eight channels have rank two and are not independent observations. This deliberately redundant background strongly influences unnormalized observation loss. Its transition law contains neither the controlled state, action nor goal. That autonomy does not imply statistical independence from chosen actions: a learned feedback controller can use nuisance observations and thereby couple its commands, and subsequent controlled states, to the background.

The observation is $o_t=[x_t,v_t,d_t]\in\mathbb R^{10}$. Its coordinates have different meanings and scales; averaging their squared errors embeds a measurement choice. The task is to approach an episode-specific target $x^\star\in[-0.8,0.8]$. Nuisance channels do not appear in the task cost.

The eight observation channels cannot reveal future nuisance innovations. Even the analytic conditional-mean reference must predict their mean, not substitute realized future values. That restriction will keep the reference causal.

We collect one hundred training trajectories, each with forty random-action transitions and an independent reset. The test set uses a different seed and a fixed task-like behavior policy. Every complete trajectory belongs to only one split. We concatenate transition pairs, not adjacent rows across resets, so the last state of one episode never becomes the first state of another.

These diagnostics describe the actual finite sample. The analytic standard deviation of 20 is a population quantity, not a claim that a sampled trajectory has exactly that standard deviation. The stable controlled process keeps the relevant coordinates bounded without clipping them at a hidden threshold. Random actions provide variation over the allowed command interval; they do not establish coverage of every state-action pair.

The observation models obtain a PCA basis from training inputs and fit a linear action-conditioned transition in that basis. We compare two and four components. The four-dimensional observation subspace contains both relevant coordinates and the two nuisance dimensions. The state-focused model fits only the first two coordinates and fills unmodeled background predictions with the training mean.

These models differ in what they represent and in parameter count. The state-focused model knows which coordinates the task cost uses. PCA knows neither the cost nor that coordinate assignment. The comparison tests the mismatch between an unweighted reconstruction objective and this declared task, not whether supervised coordinate selection is automatically available for a real robot.

A small relevant-coordinate loading supports the intended compression mechanism in this sample. It need not be exactly zero: finite-sample correlations can leak action-related information into the nuisance-dominated basis. A nonzero action coefficient alone does not establish useful action effects; we also measure predictions and control outcomes.

For a latent dimension $k$, the fitted matrices have shapes $P\in\mathbb R^{10\times k}$, $A\in\mathbb R^{k\times k}$, $B,c\in\mathbb R^k$. A batch of $N$ latent rows is advanced as $LA^\top+aB^\top+c$, giving $N\times k$ predictions. The state model has $k=2$ and receives state rows directly.

All predictors receive the current observation and action. The analytic reference uses the true controlled law and the nuisance conditional mean $0.95d_t$. It does not observe the next nuisance innovation. Its aggregate error is therefore positive (the nuisance channels contribute the residual variance of the process) even though its prediction of the noiseless relevant transition is exact.

The observed comparison is an existence demonstration. In particular, the state-focused model's mean-completion rule produces a far larger aggregate observation error than the two-component observation model (because the mean-completion residuals on the large-scale nuisance channels dominate), even though the state-focused model has the smaller relevant-coordinate error. This shows that an aggregate score dominated by decision-irrelevant channels and a task-relevant score can rank the models oppositely. The four-component model is a capacity ablation, not a failure that must be manufactured. Retaining the controlled subspace can improve both prediction and control.

All model-based controllers use random-shooting MPC with $K=200$ sequences of $H=8$ actions. They minimize the same predicted cost,

$$J=\sum_{h=0}^{H-1}\left[(\hat{x}_{h+1}-x^\star)^2+0.05\hat{v}_{h+1}^{\,2}+0.01a_h^2\right].$$

Here $\hat{x}_{h+1},\hat{v}_{h+1}$ are the model states after candidate action $a_h$, and $x^\star$ is the known episode goal. Each controller executes only the first action and replans from the next observation. The real simulator measures cost after that action, using the same coordinate weights. This keeps the prediction horizon's indexing consistent with evaluation.

A model-free PD controller provides a baseline. Its gains are tuned on separate simulator episodes, and the final comparison uses held-out episodes. The analytic MPC controller has the true dynamics, but finite-horizon random shooting is not an optimal-control solver; its cost is a reference, not a universal lower bound. Candidate action arrays and environmental resets use the same seeds across model-based controllers. The prediction model is the changed component; earlier selected actions can then produce different observations and subsequent decisions.

The two-component observation model retains little of the controlled state and its controller incurs greater cost than the state-focused model in this test. Yet completing the state-focused model's nuisance outputs with the training mean penalizes it heavily in aggregate observation error. This pair reverses the error-versus-control ordering. The four-component model is a useful counterpoint: retaining the relevant subspace can make an observation model useful for control too. There is no claim that the aggregate metric is flat across dimension counts, or that its globally best scorer must be the worst controller.

Several design choices matter. We specified a large background whose transition law is unaffected by actions; gave the state-focused learner task-coordinate knowledge; used a particular output-completion rule; and chose one goal family and planner. Changing any of these can change the rankings. In particular, adding a nuisance predictor to the state-focused model can lower its aggregate error without changing its decisions. Normalizing channels or using task-sensitive loss weights are possible alternatives, but this notebook has not trained and tested those alternatives, so it does not claim they recover a ranking.

Report coordinate-wise errors alongside the full observation score and evaluate actions in an independent environment. Good one-step predictions do not certify multi-step reliability, calibration, or robustness after a dynamics change. Conversely, an aggregate score containing coordinates outside a model's declared prediction contract may say little about its intended consumer. The example makes that distinction inspectable with code; it supplies no hardware evidence or architecture-wide conclusion.

The second experiment asks whether a factorized representation can predict combinations of factors it never saw together. We construct a transition family in which the true interaction between two context factors is multiplicative, so that an additive model is structurally wrong, and then we add a stress condition in which even the multiplicative assumption fails.

The product law supplies a specific interaction that the additive feature family cannot represent exactly across the chosen factor levels. Even when an additive law is correct, its finite-data estimation and held-out predictions can still be tested. Here we use a normalized quadratic damping term, $-\kappa v|v|$, and let the coefficient depend on two context factors; this is a declared toy law, not a universal physical drag model. The capped regime asks a separate question: how does the product-feature model behave when that law changes? A test supporting a model is not thereby rigged, but testing a violated assumption makes the boundary of the result explicit.

The system is a scalar velocity with quadratic damping:

$$v_{t+1} = v_t + \Delta t \left( a_t - \kappa(s, p) \, v_t |v_t| \right) + \epsilon_t$$

where:

• $v_t$: scalar velocity at step $t$
• $a_t$: action (normalized acceleration) at step $t$
• $\Delta t = 0.2$: integration timestep
• $\kappa(s, p)$: damping coefficient determined by the surface factor $s$ and payload factor $p$
• $s \in \{1, 2, 3\}$: surface factor level
• $p \in \{1, 2, 3\}$: payload factor level
• $\epsilon_t$: additive noise at step $t$

where $s$ is a surface factor, $p$ is a payload factor, and both take levels in $\{1, 2, 3\}$. The coefficient $\kappa$ is what we will try to learn. In the main regime,

$$\kappa_{\text{main}}(s, p) = s \cdot p$$

In the stress regime, the damping saturates:

$$\kappa_{\text{stress}}(s, p) = \min(s \cdot p, \kappa_{\max}), \qquad \kappa_{\max} = 5$$

Here $\kappa_{\max}$ is the saturation ceiling: at high $s \cdot p$ the damping stops growing, so the true law is no longer a pure product.

The stress regime specifies a capped coefficient, $\kappa = \min(sp, 5)$. This is a deliberate alternative law, not a claim that every physical damping coefficient saturates. The held-out products are 3 and 6, so the test includes uncapped and capped combinations. A pure-product feature family is therefore misspecified in this regime. The coefficients and factors here are normalized toy quantities; the experiment does not establish a physical stability margin.

Rather than fit a full transition network, we estimate the single scalar $\kappa$ for each context by least squares, then study how meta-models map context to $\kappa$. This controls the feature families and makes the scalar identifiability condition explicit. It does not remove estimation error: the finite noisy identification samples produce uncertain coefficient labels, and that uncertainty propagates into the meta-model errors alongside approximation error. Known ground truth lets us measure those errors, not declare the identification step exact. A real system may also permit scalar fitting, or may require a richer model with additional measurement and model errors.

The identification samples $(v, a)$ pairs independently rather than rolling the system forward. For this one-parameter least-squares fit, a unique coefficient estimate requires $\sum_i (\Delta t\,v_i|v_i|)^2>0$. More spread can improve information under the stated noise model, but a visual velocity range is not a general persistent-excitation test. Gaussian velocity samples have unbounded support; the code imposes no hard $|v|\leq3$ limit.

Every factor level appears in training. What is held out is specific combinations: $(1,3)$, $(2,3)$, $(3,1)$, and $(3,2)$. The training set contains the diagonal $(1,1), (2,2), (3,3)$ and two symmetric off-diagonal cells $(1,2)$ and $(2,1)$.

The structure of the split is deliberate. Every level of $s$ and every level of $p$ appears in training, so the model has seen each factor individually. What it has not seen is the specific way high $s$ combines with high $p$ and vice versa. If the factors combine additively, the model can predict the held-out cells by extrapolating each factor's effect. If they combine multiplicatively, the model needs to know the product, and the training cells do not uniquely determine the product unless the model's functional form includes it. This is the sense in which the split tests the inductive bias rather than the data coverage.

Two meta-models map context to the damping coefficient:

• Factorized (additive) baseline. $\hat{\kappa} = b + w_s s + w_p p$. No interaction term.
• Interaction-capable baseline. $\hat{\kappa} = b + w_s s + w_p p + w_{sp} s p$. Characterizes the multiplicative structure.

Both are fit by least squares on the training cells. We check the design matrix rank and condition number so we are not fitting an arbitrary solution.

These diagnostic averages track the analytic coefficients closely in the unsaturated regime. Identification noise still enters every fitted meta-model, so held-out error combines that noise, estimation variance, and functional-form mismatch. The repeated study below measures this combined variability rather than attributing every error exclusively to structure.

We repeat the whole procedure eight times with fresh identification noise, so the error bars capture variation from sampling, identification and meta-model fitting under this fixed split. They are standard errors of each model's mean, not standard deviations of individual runs. Assess a model difference against uncertainty in its estimated mean difference; it need not exceed individual-run variability. The models share each repeat's training data, so paired repeat differences would provide a direct uncertainty estimate for their comparison. The separate marginal bars below do not supply that estimate.

In the multiplicative regime the interaction model has much lower held-out coefficient error than the additive model. The table also reports error against analytic truth on seen combinations: training-side approximation error is part of this comparison, not something to assume away. The product feature represents the specified law, while the additive family cannot represent it over the full grid. Under the capped law that advantage does not persist. These results concern two chosen feature families and splits, not every factorized representation or a universal compositionality advantage.

Under saturation, the interaction model's held-out error increases, while the additive model's error decreases relative to its own multiplicative-regime result. The mean ranking reverses in this seeded study. Both remain misspecified: neither functional form represents the capped product exactly. This is not evidence that saturation always favors an additive model. The product feature has lower error under the product law here, and that advantage does not survive this cap. Misspecification alone does not establish operational unreliability: deciding whether either error is acceptable requires a task-specific tolerance. Test the target regime and report the measured errors against that requirement.

The practical implications are direct:

• Test compositional claims with held-out combinations and report seen and unseen separately.
• When interpreting recovered factor effects, establish identifiability under the chosen feature family and report its design rank. Keep this identification claim separate from measured held-out performance.
• Include a violation condition so that the reader knows what happens when the assumed structure breaks.
• Prefer a model whose inductive bias matches the regime you will deploy in. Justify that regime from known structure or an appropriately powered target-system study; a small trial need not detect rare or later changes.

A predeclared held-out comparison can be a valid test even when it supports the proposed method. Adding a violation condition asks a separate question: how does performance change when the assumed law no longer holds? The capped regime supplies that boundary check here; it does not make all positive-only studies invalid. For deployment, test the target regime and ask whether the measured errors meet its requirements rather than seeking an unconditional model ranking.

An illustrative controlled example with a known ground-truth transition family, not a benchmark result.

The third experiment compares specified cue-retention and belief-update rules, then changes the goal or corrupts the cue. It is an information-policy comparison, not a pure storage-capacity ablation: the memoryless baseline deliberately ignores the initial cue and later rewards, whereas the updating agent uses both.

To study the benefit of historical information, use a task where the current observation omits relevant information. A fully observed task can still test memory overhead, distraction or harm, but cannot establish that history is necessary. Here the current position omits the hidden goal. The initial cue and later rewards provide information about that goal, and the flip condition changes whether retained cue information remains valid.

The agent controls a scalar position $x_t$ with dynamics

$$x_{t+1} = \mathrm{clip}\!\left( 0.9\, x_t + 0.8\, a_t + \eta_t, \; [-5, 5] \right), \qquad a_t \in [-1, 1]$$

where:

• $x_t$: scalar position at step $t$
• $a_t \in [-1, 1]$: control action at step $t$
• $0.9$: position decay factor
• $0.8$: control gain
• $\eta_t$: process noise at step $t$
• $\mathrm{clip}(\cdot, [-5, 5])$: projection onto the interval $[-5, 5]$

The dynamics coefficients are shared across episodes. What differs is the goal: $x^\star=+1$ in mode 0 and $x^\star=-1$ in mode 1. The non-oracle agents do not observe the mode directly. They can receive an initial cue and subsequent noisy rewards:

• A cue at $t = 0$. A noisy reading of the goal, $\text{cue} = x^\star + \nu$ with $\nu \sim \mathcal{N}(0, 0.7^2)$. For the non-oracle agents, this is the only initial observation carrying goal information; they also observe position.
• A reward signal after each transition. $r_t = -(x_{t+1} - x^\star)^2 + \xi_t$ with $\xi_t \sim \mathcal{N}(0, 1.5^2)$. This is slow and noisy, but it is the only channel that can reveal a change.

The cue supplies evidence before the first action, but is available only at the start and can be corrupted. A cue-only agent that freezes its belief cannot correct it without another signal or update. Rewards arrive after transitions and can support later correction when their goal-conditioned likelihoods differ. At $x_{t+1}=0$ those likelihoods coincide, so that reward alone does not distinguish the two goals. The updating agent combines cue and reward evidence under its assumed model; correction speed depends on the observed evidence and chosen actions.

Part II, Chapter 2's belief-state machinery supplies a useful representation: a distribution $b_t$ over the hidden mode, updated from observations under a specified model. Its adequacy depends on that model and the decision rule. A belief representation alone does not guarantee robustness to noisy cues or changing goals; this experiment tests one particular filter and controller.

Four agents:

• Memoryless. Ignores the cue entirely and starts from a uniform prior $p(m=1) = 0.5$. It uses the prior-mean goal, which is $0$.
• Cue memory, frozen. Uses the cue to set its belief at $t=0$ and never updates it.
• Cue memory with Bayesian update. Uses the cue, then updates the belief from each observed reward, with a hazard rate of $0.05$ representing the prior probability that the context changed on any step.
• Oracle. Knows the mode.

All four share a certainty-equivalent controller: given $p=P(m=1)$, the expected goal is $\hat{x}^\star=1-2p$, and $a=(\hat{x}^\star-0.9x)/0.8$ is clipped to $[-1,1]$. The map from belief to action is fixed; access to information and belief updates differ. This minimizes the one-step expected squared error under the stated prior and zero-mean process noise when clipping is inactive, but it is not a Bayes-optimal multi-step controller. Actions can influence how informative future rewards are, and this controller does not optimize that information-gathering trade-off.

Three conditions exercise the memory in different ways:

• Stable goal, clean cue. The base case. Memory should help.
• Goal flips at $t = 10$. The cue is now stale. A memory that does not update is actively harmful.
• Corrupted cue. Each episode independently has probability 0.5 of reversing the cue-generating mean. The Gaussian cue noise remains unchanged. Under the agents' assumed clean-cue likelihood, a reversed mean can favor the wrong goal, but realized cue direction and confidence vary.

The stable condition compares the specified memories when the goal remains fixed. The flip condition introduces stale goal information; the corrupted-cue condition introduces misleading initial information in a mixture of episodes. These test two failure modes, not an exhaustive memory taxonomy. Stable-condition performance alone does not establish performance under either stress condition.

Episodes are generated with a fixed seed and shared across agents, so exogenous cue and noise draws and the initial state match across agents. Actions, subsequent states, and observed rewards can differ; sharing additive reward noise does not make reward observations identical. This common-random-numbers design aligns exogenous randomness. It can reduce paired-difference variance when the agent costs have positive covariance, but does not remove environment variance or guarantee variance reduction for every pair. Unpaired draws can add comparison noise; whether they do so depends on that covariance. Use the paired episode-cost differences when estimating uncertainty in this matched comparison.