Robotic Manipulation › Limitations and Impact

The miniature model isolates a feature-choice and rollout-evaluation problem. Translating those observations to a robot requires additional perception, actuation, contact, and safety tests.

The following are design questions, not measured performance claims about deployed manipulation systems.

The first limitation is observation realism. Our model sees exact positions. A physical system may have camera, depth, tactile, and proprioceptive measurements with occlusion, calibration error, and latency. Prediction should be tested under the actual sensing regime and material/lighting variation, not extrapolated from this state-observed toy. Reliability and evaluation methods are discussed in Part XII and Part XI. We have not measured a simulation-to-real error ratio here.

The second limitation is horizon. We evaluate only 15 steps from synthetic starts. Error rises over this window but the experiment does not establish exponential growth or any useful time limit on other models. Re-observing and replanning can reduce open-loop drift, while recurrent latent models (Part VIII) offer another design option. Their relative effectiveness must be measured on the intended task.

The third limitation is contact uncertainty. Our toy has one distance-gated interaction and no slip, rolling, or force sensing. A richer task may benefit from explicit modes, ensembles, or probabilistic predictions. None is universally preferable, and this chapter supplies no industrial deployment comparison.

The fourth limitation is task composition. A multi-stage system may hand object state between pushing, grasping, and placement components. Each interface needs evaluation because the receiving component may see inputs outside its training distribution. An affordance estimate alone does not solve this mismatch.

The central evaluation lesson is to keep prediction, uncertainty calibration, and decision quality separate. Our contact-feature regression slightly improves one-step RMSE but slightly worsens mean 15-step RMSE in this seeded run; the open-loop plan misses its target. Those results illustrate why a favorable local metric is insufficient evidence of a useful controller. Physical-robot impact would require separate task-success and safety evidence.