Frontier Questions and Research Roadmaps › Research, Engineering, and Study Roadmaps

The experiments above are local reproducibility exercises with specified mechanisms and scoped numerical performance comparisons. Everything in this section is either a defined exercise to build or test understanding, or a proposed research design seeking additional evidence. This is a distinction between these activities, not an exhaustive definition of research: a careful replication can add evidence about an already studied question. State what was tested, what was reproduced and what new claim the evidence supports.

If you are crossing into this field, consider the following learning sequence:

1. Reproduce the three experiments above and then break them deliberately. Add a third model to experiment 1 that weights channels by task relevance. Add a third factor to experiment 2 and see how the held-out error grows with the number of held-out cells. Give experiment 3 a memory that persists across episode boundaries and see what breaks.
2. Implement a small recurrent state-space model on a simple partially observed control task (Part V, Chapter 1). Compare its belief-like hidden state with a hand-built Bayesian filter under matched information and declared budgets. Differences can reflect representation, assumptions, training or optimization; the comparison does not isolate learning versus analytic filtering by itself.
3. Implement one model-based planner end to end: random shooting, then cross-entropy method (Part VII, Chapter 1). Measure how planning performance changes as you change the model, and check whether its ranking matches the prediction-error ranking. The rankings can differ; measure whether they do in your experiment.
4. Reproduce a published evaluation protocol from a benchmark you can run on one GPU or CPU, and run the leakage checks from Part XI, Chapter 5 on it. Finding a leakage issue in a published protocol is a contribution.
5. Write a transfer contract for a paper you admire and check whether the paper's claims exceed it.

This suggested sequence develops complementary skills; the steps are not strict prerequisites. Step 1 practices controlled comparisons. Step 2 compares learned and analytic beliefs. Step 3 connects model quality to measured planning quality. Step 4 practices evaluation audits, and step 5 practices claim audits. You can adapt the order to your background and available resources.

The engineering roadmap specifies artifacts to build and test for a deployment contract. Producing them can involve established engineering practice or unresolved research, depending on the system:

• A tested deployment contract. Named observation and action spaces, units, frames, rate, latency budget, and the conditions under which the system refuses to act. The refusal path is part of the contract.
• A monitored prediction stream. Running prediction error and calibration checks against live data, with drift alarms and requirement-based error thresholds where appropriate (Part XII, Chapter 2). Stationary errors can still violate a deployment requirement; a drift check alone does not test that requirement.
• A fallback controller. A reactive policy that does not depend on the model, plus a tested switch that engages it. An unexercised backup remains an unvalidated fallback candidate, not established recovery evidence.
• Scoped continual correction. Bounded update rules with a defined trigger, a defined data budget, and a rollback path (Part XII, Chapter 5).
• Resource limits. Measured latency, memory and energy over declared workloads, plus a justified bound where a worst-case guarantee is required. A maximum observed in finite trials is not the worst case over all possible inputs (Part XII, Chapter 4).
• Provenance and reproducibility. Model version, data version, environment version, seeds, and hardware recorded with every evaluation. Missing provenance can make attribution or reproduction difficult or ambiguous; controlled replications and partial records may still identify a cause.
• A documented failure catalog. Known failure cases, their operating conditions and reproductions. This is a practical recommendation, not a measured ranking of team documents.

These are readiness requirements for the proposed deployment contract. Some also pose research questions, especially for high-dimensional or safety-critical systems. An unvalidated plan alone does not establish a deployment guarantee. State which conditional properties are supported by justified proofs and which need empirical validation. Exercise the fallback under its declared trigger conditions, and support resource limits with measurements and justified analytic bounds where applicable. Neither an unvalidated estimate nor the maximum of finite trials establishes a worst-case bound.

Each of the four projects below is a proposed falsifiable design, not a completed study. Their illustrative budgets differ, and the order is thematic rather than a cost ranking. A single researcher should first reduce the task and model scope to a measured pilot budget before committing to the GPU-month or hardware estimates below.

Project 1: Decision-Aligned Evaluation MetricsLink Copied

Question and hypothesis. Does a task-weighted prediction metric predict closed-loop control quality better than an unweighted aggregate, across a family of learned dynamics models? Hypothesis: a metric that weights prediction error by the sensitivity of the task cost to each state component will rank models much more consistently with closed-loop return than uniform mean squared error, and will do so across several tasks and model classes rather than in a single constructed example.

Prior evidence. The first experiment supplies a pairwise ranking discrepancy in a controlled setting. Lambert et al. (2020) study PETS dynamics models and planning in simulated cartpole and half-cheetah, comparing expert, on-policy and state-space data distributions. In these evaluated settings, one-step prediction likelihood is not always correlated with downstream control performance; likelihood is not interchangeable with aggregate mean squared error. MBPO examines model-generated-data bias and PETS incorporates uncertainty into model-based control. The proposed study asks which candidate metrics track return over its declared task population, without claiming that no related cross-task study exists.

Strongest feasible baseline. Uniform mean squared error on the observation, and variance-normalized mean squared error. Any proposed metric must beat both.

Training and held-out population. Train five to eight model classes (linear, MLP, recurrent state-space, small transformer, two latent-variable variants) on three task families (a positioning task, a contact-rich pushing task, and a simple navigation task) in simulation. Hold out entire task instances (different mass ranges, different floor friction, different layouts), not random transitions.

Intervention. Vary the metric, holding the model family, data, and training procedure fixed. Additionally, ablate the weighting scheme: sensitivity-derived weights, learned weights, and hand-tuned weights.

Success criterion. Before evaluation, declare a practically meaningful rank-correlation improvement margin and a paired uncertainty analysis for each baseline comparison. Specify independent sampling units, seed roles, the interval or test, power assumptions, and how the multiple comparisons will be handled. Compare the metrics on the same held-out predictions and outcomes. Count a task family as supporting the hypothesis only when the predeclared analysis supports an improvement beyond the margin against both baselines; require this in at least two of the three task families. This is a proposed acceptance rule, not a result of the toy experiment.

Failure criterion. None of the tested metrics reliably improves on the baselines under the specified uncertainty analysis. This constrains those metrics and this study population. Insufficient power, poorly estimated weights and properties omitted by the metrics remain possible explanations; it does not prove that every weighted loss is inadequate.

Budget. Illustrative planning allowance: one to two GPU-months of training, roughly $10^6$ environment steps per task family, and a few CPU-months of evaluation. These are not measured requirements. A pilot must specify hardware, throughput, repetitions and monetary cost. Generated data does not remove simulator, asset or code licensing obligations.

Expected artifact. A benchmark suite with held-out instance splits, a reference implementation of each metric, and a public table of metric-versus-return rank correlations.

Leakage, safety, and privacy risks. Reusing task instances to tune and report the weighting can overstate generalization. Separate tuning from reporting instances and audit preprocessing and selection for leakage. The proposed simulation uses no human data. Confirm asset licenses and bound compute costs; this protocol provides no physical-deployment safety evidence.

A null result limits claims about these metrics on the tested tasks, subject to statistical power. Follow-up studies can test uncertainty, action-effect and rollout metrics. It does not justify a field-wide conclusion that no scalar metric can be useful.

Project 2: Cross-Embodiment Transfer Under a Declared InterfaceLink Copied

Question and hypothesis. How much of the benefit of multi-embodiment pretraining survives when the shared action interface is made explicit and the target uses a motion primitive absent from the source data? Hypothesis: shared observation encoders transfer substantially, action-effect prediction transfers only within clusters of embodiments with compatible action semantics, and negative transfer appears on the target's unseen primitive.

Prior evidence. Open X-Embodiment reports policy-transfer results from multi-robot co-training. Those results do not by themselves validate transfer of a learned dynamics interface. This project specifies an action-primitive holdout and evaluates dynamics prediction separately from policy success.

Strongest feasible baseline. A specialist model trained on target-embodiment data only, matched in parameter count and update count to the shared encoder plus adapters.

Training and held-out population. Source: a public multi-robot manipulation corpus. Target: a simulated arm with an action space that includes a primitive not present in the source (for instance, a compliant insertion primitive). Hold out the target's entire task family, not individual episodes.

Intervention. Train (a) target-only, (b) joint with a shared action interface, (c) pretrain plus frozen adapter, (d) pretrain plus fine-tuned adapter. Report action-effect prediction error separately from task success.

Success criterion. Predeclare the target-task criterion and interaction-budget comparison, a meaningful improvement margin, and a paired uncertainty analysis across the stated seeds and target-task units. Specify power assumptions and handling of comparisons across conditions. Require condition (c) or (d) to improve on (a) beyond that margin with fewer target interactions. Separately predeclare an action-effect-error tolerance and uncertainty rule for the unseen primitive; require that comparison to meet the stated noninferiority criterion. Neither a raw mean difference nor an unspecified seed-variability threshold supplies this rule.

Failure criterion. No condition beats the specialist, or the aggregate improves while the unseen primitive degrades beyond the specified uncertainty margin. The latter supports a negative-transfer finding for this protocol; publication value depends on the study's rigor and scope.

Budget. Illustrative allowance: several GPU-months for pretraining plus target fine-tuning, to be revised using a measured pilot. A simulated target avoids physical robot acquisition and operation costs, not compute-hardware costs. If a physical target is used, pilot the required robot-hours and obtain a safety review before committing to several hundred hours.

Expected artifact. A reproducible transfer benchmark with a declared interface, per-primitive error breakdowns, and adapter checkpoints.

Leakage, safety, and privacy risks. Audit the instances, categories and task families declared held out, including any source or pretraining overlap (Part VI, Chapter 4). Scene overlap violates a scene-novelty claim, but need not violate a task-family-only holdout. Document the actual split and available corpus-audit evidence. Physical deployment requires force limits and a tested fallback.

Failure to detect a transfer benefit constrains this source corpus, adapter, target population, and budget. It does not identify the cause. Follow-up ablations of action interfaces, encoder capacity, optimization, and source coverage are needed before assigning the bottleneck.

Project 3: Interventional Evidence for Learned StructureLink Copied

Question and hypothesis. When a model is trained on observational trajectories from an environment with known causal structure, how well does it predict a designated object's intervention response? Observational loss alone does not generally identify that response without resolving assumptions; randomization, valid adjustment or justified structural restrictions can change this. For a declared model and intervention population, test the hypothesis that observational score differences weakly predict intervention performance. The loss-matched comparison below separately asks whether architectures with similar observational scores differ under intervention.

Prior evidence. Schölkopf et al. survey the causal-representation research program; Ahuja et al. give conditional identification results and interventional learning procedures. This project seeks a controlled comparison of learned control models with matched observational loss. The cited work motivates that comparison but does not establish a literature-wide absence of similar studies.

Strongest feasible baseline. Observational-loss-matched models with different architectures: an object-centric slot model, a plain latent state-space model, and a pixel-level predictor. All tuned to comparable held-out prediction loss.

Training and held-out population. A simulated scene with controllable objects. Train on trajectories from a fixed set of action policies. Hold out object configurations that require understanding an interaction the policies never produced (for instance, stacking after training only on pushing).

Intervention. Execute a controlled intervention (apply a known impulse to object $k$) in the simulator, and compare the model's predicted post-intervention trajectory against the realized one. Repeat for each object and each of several impulse magnitudes.

Success criterion. At least one architecture achieves interventional error low enough to support planning (concretely: MPC using the model reaches the post-intervention target at a rate within a predeclared noninferiority margin of the planner with simulator access, using a confidence interval with adequate power), while an observational-loss-matched baseline does not.

Failure criterion. No architecture meets the predeclared planning threshold, or any observed architecture advantage is smaller than the study can resolve. Matched interventional errors alone do not distinguish uniformly good from uniformly bad models. Tight observational/interventional correlation would challenge the proposed weak-association hypothesis in this population, not support it.

Budget. Single-GPU scale for a simple 2D or low-resolution 3D scene; one to two GPU-months.

Expected artifact. An intervention test suite, plus a table of observational loss against interventional error for each architecture.

Leakage, safety, and privacy risks. The intervention must be applied in the environment, not in the model, or the test is circular. The held-out configurations must not appear in any training data, including pretraining data if any is used.

If observational loss tracks interventional error in this study, report that association within the tested model families, interventions and task population. It does not remove theoretical non-identifiability or establish the same association outside those conditions.

Project 4: Memory Under Changing Context at Deployment ScaleLink Copied

Question and hypothesis. In a deployment with an unknown, slowly changing context, does a memory system with an explicit change model outperform both a memoryless system and a memory system with a fixed assumed context? Hypothesis: adaptive memory lowers post-change recovery cost under the declared change population, while a misspecified hazard can erase that advantage or make memory worse than the memoryless baseline. The performance comparisons below test this scoped hypothesis; they do not establish that every benefit is causally mediated by detection accuracy.

Prior evidence. The third experiment demonstrates stale-context behavior in a two-state toy. Kirkpatrick et al. study parametric-network forgetting in sequential permuted-MNIST classification and Atari learning, comparing elastic weight consolidation with their stated baselines. Dohare et al., Nature, 2024 study loss of plasticity in Continual ImageNet, class-incremental CIFAR-100 and simulated ant locomotion with PPO, including changing-friction and stationary conditions. These evaluated learning failures are distinct from a stale deployment belief. The proposed project tests its declared memory/change-model comparison; those citations do not establish that comparable control studies are absent.

Strongest feasible baseline. Memoryless with matched parameters, and memory with a fixed context (the "frozen" agent in the toy, scaled up).

Training and held-out population. A simulated or physical manipulation task with changing payload, friction, or camera calibration. Evaluate on change events the system has never seen: changes in a parameter outside the training range of the change model.

Intervention. Vary the hazard rate and the change-detection signal (reward prediction error, prediction error on the observation, an explicit change classifier). Include a version with no update.

Success criterion. Predeclare the post-change cost window, unseen-event population, practically meaningful improvement margins, and a powered paired uncertainty analysis over the stated runs and events. Specify seed roles, sampling units, and handling of the two baseline comparisons. Require the analysis to support lower recovery cost beyond the declared margin against both baselines. This is an acceptance rule for the proposed study, not a guarantee for every event.

Failure criterion. The adaptive memory is not better, or the ranking depends on the change magnitude in a way that reverses. A reversal would show that this tested update rule is not uniformly best across these changes; it would not prove that no single hazard can be safe.

Budget. Simulated: one to two GPU-months. Physical: several hundred robot-hours plus a safety review and a tested fallback controller.

Expected artifact. A change-detection benchmark with unseen change events, plus recovery-cost curves as a function of change magnitude.

Leakage, safety, and privacy risks. Any memory that stores episodes must have a defined retention policy and a reset boundary to avoid carrying personal or stale data across sessions (Part XII, Chapter 3). Physical deployments must bound force and speed during the recovery window, when the belief is wrong.

Failure to detect an adaptive-memory advantage constrains this update rule and test population. Detection quality, memory representation, control policy, change frequency, and statistical power remain possible explanations; ablate them separately before naming a bottleneck.

The table is a summary, not a substitute for the specifications or measured pilot costs. Project 1 scores fixed predictions on the same data using different metric functions; their ranking differences are conditional on that metric comparison. Project 2 compares whole transfer procedures, including pretraining data, update budgets and adapters. Interface-specific attribution needs a matched interface ablation. Project 3 matches a selected observational score but does not control every training or representation confound. Project 4 tests change-model transfer on unseen events under its stated protocol. Held-out units define the generalization target and limit overlap; matched interventions, budgets and procedures address other confounds. Generalization claims need a specified evaluation population and appropriate splits. Descriptive or randomized within-population studies can still be interpretable without a train/test holdout.