Model Merging › TIES Merging

Uniform task arithmetic adds all task vectors together without accounting for conflicts. When two task vectors push the same parameter in opposite directions, the result can be worse than either individual model. TIES merging (Trim, Elect Sign, and Disjoint Merge) introduces three structured steps to handle these conflicts systematically, producing merged models that are more stable and reliable than naive task addition.

The method was introduced by Yadav et al. in their 2023 paper "Resolving Interference When Merging Models." The paper's central insight is that interference between task vectors is the primary cause of performance degradation in multi-task merges, and that this interference can be substantially reduced through a principled three-step process applied before the final combination.

To understand why TIES is necessary, consider what happens when you add two task vectors that conflict. Suppose model $A$ was fine-tuned to be more formal and model $B$ was fine-tuned to be more casual. For some parameter $j$ controlling the formality dimension of the model's output, $\tau_A[j] > 0$ (push toward formal) and $\tau_B[j] < 0$ (push toward casual). Simple addition gives $\tau_A[j] + \tau_B[j]$, a value that may be small in magnitude because the two large signals cancelled. The merged model ends up with a parameter that was pushed hard in opposite directions and lands somewhere that neither task wanted: not formal, not casual, but incoherently in between.

More concretely, consider a single parameter $\theta_j$ in the pretrained model. After fine-tuning on task $A$, it shifts by $\delta_j^A$, and after fine-tuning on task $B$, by $\delta_j^B$. If $\delta_j^A > 0$ and $\delta_j^B < 0$, simple addition gives $\delta_j^A + \delta_j^B$, a value reduced in magnitude that incorporates conflicting signals. The interference is destructive: you lose the signal from both tasks without gaining a sensible compromise.

TIES addresses this through a structured three-step pipeline applied to the task vectors before combination.

Many parameters in a task vector have very small magnitudes, representing fine-tuning noise rather than meaningful task-specific adaptation. These small values accumulate from gradient descent finding locally optimal adjustments that happen to be numerically nonzero but carry little semantic signal. When you sum multiple task vectors, these small noisy values add up and can overwhelm the large, meaningful signals that encode task-specific behavior.

The trim step removes this noise by keeping only the top-$k\%$ of parameters by absolute magnitude and setting the rest to zero:

$$\hat{\tau}_i[j] = \begin{cases} \tau_i[j] & \text{if } |\tau_i[j]| \geq \gamma_i^{(k)} \\ 0 & \text{otherwise} \end{cases}$$

where:

• $\hat{\tau}_i[j]$: the trimmed task vector value at position $j$ for model $i$
• $\tau_i[j]$: the original task vector value at position $j$ for model $i$
• $\gamma_i^{(k)}$: the $k$-th percentile threshold of $|\tau_i|$, the magnitude threshold computed separately for each model

A typical setting keeps the top 20% of values (i.e., $k = 20$), discarding the bottom 80% as noise. This sparse representation removes clutter before the merge. The intuition is that the parameters with the largest magnitude changes are the ones the optimizer found most useful for the task. The parameters with small magnitude changes were essentially unchanged by fine-tuning and contribute more noise than signal when merged.

The trim step turns each dense task vector into a sparse one: 80% zeros and 20% nonzero values. This sparsity is what allows the next step to reason clearly about conflicts.

After trimming, each parameter position may have nonzero values from multiple task vectors, potentially with opposing signs. For a parameter that two tasks disagree about, you need a principled way to decide which direction to trust. The elect sign step resolves this by choosing a "majority vote" direction for each parameter, weighted by magnitude.

For each parameter position $j$, compute the total positive and negative momentum across all task vectors:

$$m^+[j] = \sum_{i=1}^{k} \mathbb{1}[\hat{\tau}_i[j] > 0] \cdot |\hat{\tau}_i[j]|$$ $$m^-[j] = \sum_{i=1}^{k} \mathbb{1}[\hat{\tau}_i[j] < 0] \cdot |\hat{\tau}_i[j]|$$

where:

• $m^+[j]$: total magnitude of positive updates at position $j$ across all models
• $m^-[j]$: total magnitude of negative updates at position $j$ across all models
• $\mathbb{1}[\cdot]$: indicator function (1 if condition holds, 0 otherwise)

The elected sign for position $j$ is:

$$\gamma[j] = \text{sign}(m^+[j] - m^-[j])$$

where ties can be broken arbitrarily, often with a small positive bias. This elected sign vector $\gamma$ represents the consensus direction: the direction that the majority of task vectors (weighted by magnitude) want to push each parameter.

The magnitude weighting is important. A single model with a large, confident update in one direction should outweigh several models with small, uncertain updates in the opposing direction. Simply counting votes without weighting by magnitude would give equal voice to a highly confident update and a near-zero noisy one. The weighted vote ensures that the consensus direction reflects where the most important adaptations agree.

With elected signs in hand, the final merge discards any task vector contributions that conflict with the consensus:

$$\tau_{\text{merged}}[j] = \frac{1}{|\mathcal{A}[j]|} \sum_{i \in \mathcal{A}[j]} \hat{\tau}_i[j]$$

where:

• $\mathcal{A}[j] = \{i : \hat{\tau}_i[j] \neq 0 \text{ and } \text{sign}(\hat{\tau}_i[j]) = \gamma[j]\}$: the set of models with nonzero values agreeing with the elected sign at position $j$
• $|\mathcal{A}[j]|$: the number of models in agreement at position $j$

Only task vectors with the consensus sign contribute to the average at each position. Models that disagree are silenced for that parameter. The average is taken only over the agreeing models, not over all $k$ models, so disagreeing models do not dilute the signal by contributing zeros to the denominator.

The final merged model is:

$$\theta_{\text{merged}} = \theta_{\text{pretrained}} + \lambda \cdot \tau_{\text{merged}}$$

This three-step process is more conservative than simple task arithmetic. By filtering small values and resolving sign conflicts before averaging, TIES produces merged models with less destructive interference, particularly when merging three or more models with overlapping parameter sensitivities.

The three steps address distinct sources of noise in a principled sequence. Trimming removes small, incidental changes that fine-tuning made but that carry little semantic meaning. Sign election identifies the consensus direction for parameters that multiple tasks care about. Disjoint merge combines only the aligned contributions, avoiding the cancellation that would occur if opposing signals were averaged together.

Consider a worked example with three models merging a parameter that controls formality of output. Model A pushes it strongly positive (formal). Model B pushes it moderately positive. Model C pushes it weakly negative. After trimming, suppose model C's small negative value is discarded as noise. Sign election sees two positive votes and zero negative votes: the elected sign is positive. Disjoint merge averages model A's and model B's contributions. The result is a clearly positive adaptation toward formality, which is what two of the three models confidently wanted.