Frontier Questions and Research Roadmaps › Summary

Three questions, three experiments and four proposed projects connect capability claims to specified evidence. The examples show why prediction, intervention, control and recovery measurements need separate interpretations; they do not rank the field's unresolved constraints.

Generality has to be written as a contract. State the source and target systems, query semantics, observation and action spaces, held-out population, adaptation budget and criterion. Shared interfaces need demonstrated competence. Specialists with matched target data and budgets are informative baselines. Ambiguous action mappings, unobserved context, capacity constraints and changed calibration are possible transfer failures, not an exhaustive or ranked list of causes.

Structure has to be earned. Observational data alone does not generally identify interventions or counterfactuals without additional assumptions. Suitable adjustment, randomization, or structural restrictions can change what is identifiable. Causal claims require stated assumptions and sufficient identifying information. Intervention variation is needed by some approaches, while others can identify effects from observational data under justified assumptions; independent intervention tests can check the resulting predictions. A disentangled latent or a named object slot alone does not establish a causal effect. For empirical unseen-combination performance, declare the held-out split; when interpreting recovered factor effects, check identification under the stated model family and assumptions. An assumption-violation condition separately probes the limits of a matched inductive bias. Persistent memory includes several mechanisms. In the third experiment's goal-flip condition, frozen cue memory has higher mean cost than the memoryless baseline; that ranking is conditional on this task and controller.

Evaluation has to reach the decision. One-step loss, rollout error, calibration, interventional prediction, action ranking, closed-loop return, safety outcomes, latency and adaptation cost answer different questions. A measurement of one does not guarantee another. The evaluator and evaluation population can depend on the model, so use splits at the trajectory or task level, audit for leakage, and do not treat seeds as independent systems.

The three experiments, restated as conditional examples to investigate on your own models:

• In experiment 1, the two-component observation predictor has lower aggregate observation error but higher controller cost than the state-focused predictor. Report both coordinate-specific error and true-environment decisions.
• In experiment 2, the product-feature model has lower held-out error under the multiplicative law, but not under the capped law. Report seen and held-out errors for both feature families and regimes.
• In experiment 3, fixed cue memory improves on the memoryless baseline in the stable condition, but has higher cost in the tested change and corrupted-cue conditions. Report both stable and changed conditions.

Reproduce the experiments and test conditions that challenge their assumptions. Write transfer contracts for the papers you read and check whether their claims stay within them. Build the deployment artifacts, especially the failure catalog and tested fallback. For a proposed study, declare the evaluation population and failure criterion before collecting data. A rigorous null can constrain a stated hypothesis; an unfalsifiable positive claim cannot supply that test.

A well-powered null result constrains the hypothesis tested under the declared protocol; it does not prove that no effect exists. The four projects should make either outcome informative. Declare the comparison and failure criterion, run the study, and use the evidence to decide what to test next.