Pretraining, Post-Training, Scaling, and Adaptation › Task adaptation and instruction conditioning

Pretraining gives you a general model. Adaptation makes that model specific. The central question of adaptation is how to inject task information into a pretrained model without destroying what the pretraining learned. There are three broad families of answers: full fine-tuning, parameter-efficient fine-tuning, and conditioning. Each family represents a different bet about how much of the pretrained model needs to change for the target task, and each has different costs in memory, compute, and risk of forgetting.

The naive approach is to take a pretrained model and continue training it on data from the new task. This works, but it has two well-known problems.

Catastrophic forgetting. Target-only updates can reduce performance on previously learned tasks, especially when their data are absent during adaptation. The amount of forgetting depends on task similarity and the training protocol; it should be measured on retained source evaluations rather than inferred from target performance alone. Methods such as elastic weight consolidation were developed to reduce this failure mode.

Sample efficiency. Full fine-tuning exposes every weight to the target loss. With a small target dataset, this gives the optimizer many ways to fit idiosyncrasies rather than reusable structure. Whether overfitting occurs depends on the initialization, regularization, learning rate, and validation protocol, but early stopping and source-retention checks are especially important in this regime.

Both problems motivate restricting which parameters are updated. The natural question becomes: how much of the model needs to change? If the answer is "much less than all of it", then the two problems above collapse into a much smaller version of themselves.

The idea behind parameter-efficient fine-tuning (PEFT) is to update a small number of parameters while keeping the pretrained backbone frozen. A widely used method is LoRA (Hu et al. 2021). For a frozen linear layer with weight matrix $W_0 \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$, LoRA adds a low-rank update:

$$W = W_0 + \frac{\alpha}{r} B A$$

where:

• $W_0 \in \mathbb{R}^{d_{\text{out}} \times d_{\text{in}}}$: the frozen pretrained weight matrix
• $W$: the adapted weight matrix
• $d_{\text{in}}$: the input dimension of the frozen layer
• $d_{\text{out}}$: the output dimension of the frozen layer
• $A \in \mathbb{R}^{r \times d_{\text{in}}}$: the first trainable low-rank factor
• $B \in \mathbb{R}^{d_{\text{out}} \times r}$: the second trainable low-rank factor
• $r \ll \min(d_{\text{in}}, d_{\text{out}})$: the rank of the update
• $\alpha$: a scaling constant
• $\frac{\alpha}{r} B A$: the low-rank update added to the frozen weights

Only $A$ and $B$ are updated. The forward pass becomes

$$h = W_0 u + \frac{\alpha}{r} B A u$$

where:

• $h$: the output of the adapted linear layer
• $u$: the input vector to the linear layer
• $W_0 u$: the frozen pretrained transformation
• $\frac{\alpha}{r} B A u$: the low-rank adaptation applied to the input

LoRA assumes that a useful weight update can be represented at low rank. Hu et al. (2021) found this parameterization effective for adapting large language models, but it is not guaranteed for every layer or domain. A physical system may vary along only a few latent factors, such as mass or damping, yet a nonlinear network can still require a higher-rank weight change to represent their effect. Rank is therefore an empirical capacity choice, not a consequence of the physical parameter count.

Other PEFT methods include:

• Adapters: small bottleneck MLPs inserted between frozen layers.
• Prefix and prompt tuning: learnable vectors prepended to the input or to attention keys and values.
• Bias-only tuning: updating only the bias terms.
• IA$^3$ and related methods: scaling the activations of frozen weights by learned vectors.

Each has different tradeoffs in expressiveness, memory, and inference latency. LoRA is a common default because it adds no inference latency when merged back into $W_0$. Adapters are easy to stack and interpret, prefix tuning avoids touching the weight matrices entirely, and bias-only tuning is the cheapest of all but also the least expressive. In practice, the choice between them is often dominated by the deployment constraints: if you need to serve dozens of adapted variants from a single base model, LoRA's ability to merge into the base weights or sit as a small sidecar is usually decisive.

A complementary approach is to condition the model on task information rather than modifying its weights. This is what language models do when they are prompted. The model is unchanged, but the input carries a description of what to do. Conditioning is appealing because it can be applied without any gradient updates at all, provided the pretrained model has learned to interpret the conditioning signal during pretraining.

For world models, instruction conditioning comes in several flavors.

• Task identifiers. A learned embedding per task, concatenated to the model's input. This is the simplest form, and it is what our worked example uses.
• Natural language instructions. A text description such as "this robot has a soft gripper and a heavy base", encoded and used as a conditioning vector. This can describe a new task without fitting a task ID, provided pretraining taught the model how those descriptions relate to dynamics.
• Goal specifications. A target state, a set of constraints, or a reward function, passed to the model as input. This is the foundation of goal-conditioned policies and goal-conditioned world models.
• Demonstrations. A few example transitions in the context window. This is in-context learning, and it is the most flexible form of adaptation because it requires no weight updates at all.

Instruction conditioning shifts work into pretraining. The model must see enough paired descriptions and dynamics to learn how the signal should affect predictions. If the signal omits a task-relevant distinction, two different systems become indistinguishable to the predictor. Conversely, a high-capacity descriptor is useful only when the data identify how each part should be used; otherwise the model may ignore it or fit spurious correlations. Generalization to a newly described task must be measured rather than assumed.

Another approach is meta-learning: training an initialization so that a small number of gradient steps can adapt it to a task. The classic formulation is MAML (Finn et al. 2017), which optimizes the initialization through one or more task-specific gradient updates. For world models, this directly targets a regime where interaction data is scarce, provided the meta-training task distribution represents the deployment tasks.

Meta-learning depends on the distribution of tasks used during meta-training. A target outside that distribution can erase its advantage over ordinary fine-tuning. Original MAML differentiates through the inner adaptation step, while first-order variants approximate or omit those second-order terms. The general lesson is that rapid adaptation is itself learned under a task distribution and must be evaluated under the shifts expected at deployment.