Model Merging › Task Arithmetic

Weight averaging treats all models symmetrically and mixes them in the original weight space. Task arithmetic takes a different approach by working with weight differences, called task vectors, and operating in the space of changes to the pretrained model. This reframing supports more operations: you can combine capabilities, remove them, transfer them across contexts, and reason about them algebraically.

The idea was introduced by Ilharco et al. in their 2023 paper "Editing Models with Task Arithmetic." The core innovation is defining a task vector as the difference between a fine-tuned model's weights and the original pretrained weights:

$$\tau_{\mathcal{T}} = \theta_{\text{finetuned}} - \theta_{\text{pretrained}}$$

where:

• $\tau_{\mathcal{T}}$: the task vector for task $\mathcal{T}$
• $\theta_{\text{finetuned}}$: weights of the model fine-tuned on task $\mathcal{T}$
• $\theta_{\text{pretrained}}$: weights of the original pretrained model, kept frozen as a reference

The task vector captures everything the fine-tuning process learned. It is a signed, dense vector of the same dimension as the model parameters. Values near zero indicate parameters that the fine-tuning process left largely unchanged; large positive or negative values indicate parameters that were significantly updated to accommodate the new task.

The shift from thinking about absolute weights to thinking about weight differences is subtle but important. Absolute weights are tied to the specific scale and initialization of the pretrained model, making them hard to interpret. Weight differences are interpretable as directions in parameter space: the task vector points from the pretrained model to the fine-tuned model, encoding the specific adaptation the fine-tuning process made. By separating the shared pretrained representations (encoded in $\theta_{\text{pretrained}}$) from the task-specific adaptations (encoded in $\tau_{\mathcal{T}}$), task arithmetic enables compositional reasoning that simple weight averaging cannot.

What makes task vectors powerful is that they support a range of meaningful arithmetic operations.

Task addition lets you combine capabilities from multiple fine-tuned models into one. Given task vectors $\tau_1, \tau_2, \ldots, \tau_k$ from $k$ different tasks, you construct a merged model by:

$$\theta_{\text{merged}} = \theta_{\text{pretrained}} + \lambda \sum_{i=1}^{k} \tau_i$$

where:

• $\lambda$: a scaling coefficient that controls the overall magnitude of the task vector sum, typically tuned on a small validation set, often around 0.3 to 0.5
• $\tau_i$: the task vector for the $i$-th fine-tuned model
• $\theta_{\text{pretrained}}$: the frozen pretrained base model weights

The scaling coefficient $\lambda$ is important because stacking multiple task vectors can push the model far from the pretrained point. Too large a sum may degrade general capabilities; the scaling factor keeps the merged weights in a region where the pretrained model's priors still hold. Think of $\lambda$ as a dial that controls how much the merged model commits to the new tasks versus how much it retains from the general pretrained capabilities. In the original paper, $\lambda = 0.5$ worked well across a range of experiments, but the optimal value varies with the number of tasks and their degree of overlap.

Task negation is an equally interesting operation. By subtracting a task vector from the pretrained model, you can approximately remove a capability:

$$\theta_{\text{modified}} = \theta_{\text{pretrained}} - \lambda \tau_{\mathcal{T}}$$

If a task vector captures "how to produce toxic output," subtracting it nudges the model away from that behavior without requiring additional training data or fine-tuning. The mechanism is not perfectly precise: you are subtracting the entire change that fine-tuning made, which includes both the targeted behavior and any correlated changes. But empirically, negation has shown promise for capability removal and for detoxification. It represents a form of post-hoc editing that is dramatically cheaper than retraining.

Task analogy follows the same spirit as word vector arithmetic. Just as word embeddings allow the operation king minus man plus woman to approximate queen, we can construct analogical operations in model space. Take a model fine-tuned on French sentiment analysis, subtract the French language translation task vector, and add the Spanish language translation task vector. The result might approximate a model that performs sentiment analysis in Spanish, without ever collecting Spanish sentiment training data. These analogies do not always succeed, and their reliability depends on how semantically similar the task vectors are, but they have shown empirical promise for low-resource task transfer.

The success of task arithmetic reflects a deeper regularity in how large pretrained models encode knowledge. Pretrained language models develop internal representations that are modular and reusable across tasks. When you fine-tune on task $A$, the fine-tuning process adjusts a relatively concentrated subset of parameters substantially; most parameters change only slightly. When you fine-tune on task $B$, a different subset shifts. Because many tasks use different "slots" in the model's representational space, their task vectors have limited interference when summed.

This modularity is not guaranteed by architecture; it emerges from training. Models trained on sufficiently diverse data develop the capacity to separately encode different capabilities, and fine-tuning exploits this structure by updating only the relevant modules while leaving others approximately unchanged. The task vector essentially identifies which modules a task relies on and how they should be adjusted.

This also explains when task arithmetic fails. Tasks that require conflicting parameter adjustments will interfere destructively when added. If both fine-tunes heavily modify the same layers and those modifications are in opposing directions, the sum will be noisy and the merged model will struggle on both tasks. The later methods in this chapter, TIES and DARE, address this interference problem systematically.

The idea of task vectors connects to a broader thread in machine learning research about the geometry of parameter space. Prior work on linear mode connectivity (Frankle et al., 2020) showed that models fine-tuned from the same initialization are connected by nearly loss-flat linear paths. Work on model editing (Mitchell et al., 2021) showed that specific factual associations in language models can be located and modified through targeted weight updates. Task arithmetic unified and extended these observations by showing that the difference between pretrained and fine-tuned weights is a semantically meaningful object that can be manipulated independently of the absolute weights.

The success of task arithmetic also connects to the lottery ticket hypothesis (Frankle and Carlin, 2019), which suggested that neural networks contain sparse subnetworks that do most of the work. Task vectors implicitly identify which parameters matter for a given task, and operations like DARE (discussed later) exploit this by pruning most of the task vector and keeping only the most important parameters.