Pretraining, Post-Training, Scaling, and Adaptation › Pretraining, Post-Training, Scaling, and Adaptation

Real-world models are rarely trained once on one task from one data source. They are built in stages. A large pretraining phase consumes heterogeneous data across environments, embodiments, and modalities. A post-training phase reshapes the model so its rollouts support the decisions an agent will make. An adaptation phase specializes the pretrained model to a particular robot, task, or user, often with only a small amount of new data. This chapter is about that pipeline.

Why use a pipeline at all? Training a separate model for every application repeats representation learning and prevents data sharing. Pretraining can amortize that cost when source and target systems share useful structure. Post-training then changes what the model is optimized to do, while adaptation fits the remaining target-specific differences. Keeping those operations conceptually separate also makes failures easier to diagnose.

Suppose a team has simulation, passive video, and a small amount of target-robot interaction. A staged workflow lets it test three separate questions: whether the broad mixture learns a transferable representation, whether a control-oriented objective improves decisions, and how much target data adaptation needs. Source diversity helps only when it contains structure relevant to the target, so each stage needs its own held-out evaluation.

The three stages have different objectives and different failure modes.

• Pretraining is a data-hungry, compute-hungry phase. Its job is to absorb broad structure: how objects move, how contact works, how different sensor modalities relate to one another. Its failure modes are data contamination, memorization of spurious correlations, and evaluation that rewards surface-level fit rather than transferable structure.
• Post-training is a preference-and-goal-driven phase. Its job is to shape a general predictor into a model whose rollouts are useful for control, not merely accurate on average. Its failure modes are reward hacking, distribution collapse, and loss of the very diversity that made the pretrained model valuable.
• Adaptation is a specialization phase. Its job is to make the model work on a specific target distribution with few samples. Its failure modes are overfitting, catastrophic forgetting, and negative transfer when the target is far from the pretraining distribution.

Each stage optimizes a different quantity. Pretraining minimizes average prediction error over a broad data mixture. Post-training minimizes a scalar downstream signal, such as reward, cost, or preference. Adaptation minimizes error on a narrow target distribution using few samples. These quantities can conflict: a model optimized for one stage may be suboptimal for another.

The stages are not always separated in practice. Some models are pretrained and adapted directly; others undergo reinforcement-learning post-training without later adaptation. The conceptual separation still matters because the objectives differ. A post-trained model can have worse held-out prediction error yet produce better closed-loop decisions, so both quantities need to be measured.

This chapter develops the vocabulary and design tradeoffs needed to reason about the pipeline. It covers scaling, task conditioning and adaptation, preference and control-oriented post-training, and transfer across embodiments. A worked implementation then pretrains a small task-conditioned dynamics model, adapts it to a new system with a few trajectories, and demonstrates a preference loss.

The material builds on Part VI: Learning World Models at Scale and prepares you for Part VII: Planning and Agency, where an optimizer consumes the model's predictions.