Model Merging: Weight Averaging, Task Arithmetic, TIES, DARE

What if you could combine two separately trained language models, each expert in different domains, into a single model that inherits skills from both, without any additional training? That is the promise of model merging, a family of techniques that combine the weights of multiple models into one. Unlike ensemble methods that run several models in parallel, model merging produces a single model with the same architecture and inference cost as each individual component. You get multi-task capability with single-model overhead.

Model merging has become an important tool in the modern LLM toolkit. It enables practitioners to blend capabilities without data, without gradient computation, and often without access to the original training pipelines. You might merge a reasoning-specialized model with a coding-specialized model, or average dozens of checkpoints from different stages of a training run to find a set of weights that generalizes better. The results can be surprising: merged models frequently outperform any single component on benchmarks that span multiple capabilities. On the Open LLM Leaderboard maintained by Hugging Face, merged models have repeatedly ranked at the top, sometimes outperforming models with significantly more parameters and substantially more training compute.

The economics of model merging explain its appeal. Fine-tuning a 7-billion parameter language model from a strong base checkpoint costs a fraction of pretraining, and releasing fine-tuned variants has become common practice in the open-source community. When dozens of fine-tunes of the same base model are publicly available, each specialized for a different skill or domain, merging them becomes an attractive way to assemble a generalist model at nearly zero cost. The alternative, collecting data for all desired skills and running a joint fine-tune, requires coordination, data curation, and compute that most practitioners do not have access to.

This chapter covers the four core techniques: weight averaging, task arithmetic, TIES merging, and DARE merging. We will build intuition for why each approach works, examine the mathematical operations involved, trace through the history of how each idea emerged, and implement them from scratch. Along the way, we will pay close attention to when each method is likely to succeed and where it fails.

The simplest form of model merging is weight averaging, sometimes called model soup. Given two or more models that share the same architecture, you compute the element-wise average of their weight tensors. No gradients, no data, no hyperparameter search beyond which models to include. The operation is so simple it seems like it should not work, and yet it often does.

Formally, given $n$ models with weight vectors $\theta_1, \theta_2, \ldots, \theta_n$, the merged model has weights:

where:

• $\theta_i$: the full vector of parameters (flattened weights of all layers) for model $i$
• $n$: the number of models being merged
• $\theta_{\text{merged}}$: the resulting merged weight vector

This is a surprisingly effective baseline. The intuition is that if each model has landed in a different region of the loss landscape after training from the same or similar initialization, averaging their weights can land in a region that inherits good properties from all of them. You are not picking a single best model; you are constructing a new model that sits in the middle of the space spanned by all the candidates.

The key insight is that modern large language models are typically fine-tuned from the same pretrained checkpoint. This shared starting point means all fine-tuned models begin in the same region of weight space. Each fine-tuning run makes relatively small, task-specific adjustments to the pretrained weights. The fine-tuned model is not a radically different object from the pretrained model; it is the same model with modest parameter perturbations applied on top. The resulting models are therefore close to each other in weight space, and their average is also likely to be near all of them.

This property, sometimes called linear mode connectivity when demonstrated rigorously, means that the loss surface along the linear path connecting two fine-tuned models is relatively flat. Averaging does not catastrophically fall off a cliff. The merged model sits in a region that all component models agree is reasonable, and it can generalize across their respective tasks because the fine-tuned behavior of each model is still approximately encoded in the merged weights.

Understanding why this connectivity holds requires thinking about what fine-tuning does to a large pretrained model. A 7-billion parameter model trained on hundreds of billions of tokens has developed internal representations that are dense, distributed, and redundant. No single parameter encodes a single concept. When fine-tuning on, say, code generation, the optimizer adjusts a distributed pattern across many parameters to shift the model's output distribution toward code. These shifts are modest relative to the scale of the pretrained weights. The average of two such modest shifts still preserves the pretrained model's core representations and typically satisfies both fine-tuning objectives reasonably well.

When models are trained from entirely different initializations, however, the picture changes. The same parameter index in model $A$ and model $B$ may encode completely different things, so averaging them destroys information without creating anything useful. Weight averaging requires that models share the same parameterization and, ideally, the same initialization. This is a hard requirement, not a soft one.

The model soup approach, introduced by Wortsman et al. in their 2022 paper "Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time," takes simple averaging further by being selective about which models to include. Rather than averaging everything uniformly, you add models to the soup one at a time, keeping a model only if it improves the merged model's validation performance.

The procedure is:

1. Sort candidate models by their individual validation accuracy.
2. Start with the single best model as the initial soup.
3. Attempt to add the next-best model by averaging it into the current soup.
4. If the resulting soup performs better on validation, keep it; otherwise discard the candidate.
5. Continue until all candidates have been evaluated.

This greedy selection tends to produce better results than uniform averaging because it filters out models that move the soup in harmful directions. In practice, the candidate models might be checkpoints from different training runs, different hyperparameter configurations, or different amounts of fine-tuning on the same base model. Wortsman et al. demonstrated that model soups could achieve better accuracy and robustness than the best individual model, even outperforming ensembles of the same models in some settings, a remarkable result because inference cost is identical to running a single model.

The mechanism behind improved robustness is particularly interesting. Individual fine-tuned models often overfit to the specific details of their fine-tuning setup: the augmentation strategy, the learning rate schedule, the random seed. Averaging multiple such models smooths out these idiosyncrasies, producing a model that generalizes better to distribution shifts. The soup is less adapted to any single training artifact and more aligned with the shared signal across all models.

A natural extension of uniform averaging is weighted averaging, where different models contribute unequally to the final result:

where:

• $w_i$: the weight assigned to model $i$, satisfying $\sum_{i=1}^{n} w_i = 1$ and $w_i \geq 0$
• $w_i$ can reflect validation performance, recency (later checkpoints weighted more), or domain relevance

Uniform averaging assigns $w_i = 1/n$ to all models. Weighted averaging lets you express preferences. If one model substantially outperforms others on the target domain, you might assign it a higher weight. If you trust one training run's hyperparameters more than another's, that run can contribute more to the final soup. Finding optimal weights is a convex optimization problem given a validation set, but the space of weights is large, and practitioners often default to simple heuristics like proportional assignment based on validation accuracy.

Weight averaging treats all models symmetrically and mixes them in the original weight space. Task arithmetic takes a different approach by working with weight differences, called task vectors, and operating in the space of changes to the pretrained model. This reframing supports more operations: you can combine capabilities, remove them, transfer them across contexts, and reason about them algebraically.

The idea was introduced by Ilharco et al. in their 2023 paper "Editing Models with Task Arithmetic." The core innovation is defining a task vector as the difference between a fine-tuned model's weights and the original pretrained weights:

where:

• $\tau_{\mathcal{T}}$: the task vector for task $\mathcal{T}$
• $\theta_{\text{finetuned}}$: weights of the model fine-tuned on task $\mathcal{T}$
• $\theta_{\text{pretrained}}$: weights of the original pretrained model, kept frozen as a reference

The task vector captures everything the fine-tuning process learned. It is a signed, dense vector of the same dimension as the model parameters. Values near zero indicate parameters that the fine-tuning process left largely unchanged; large positive or negative values indicate parameters that were significantly updated to accommodate the new task.

The shift from thinking about absolute weights to thinking about weight differences is subtle but important. Absolute weights are tied to the specific scale and initialization of the pretrained model, making them hard to interpret. Weight differences are interpretable as directions in parameter space: the task vector points from the pretrained model to the fine-tuned model, encoding the specific adaptation the fine-tuning process made. By separating the shared pretrained representations (encoded in $\theta_{\text{pretrained}}$) from the task-specific adaptations (encoded in $\tau_{\mathcal{T}}$), task arithmetic enables compositional reasoning that simple weight averaging cannot.

What makes task vectors powerful is that they support a range of meaningful arithmetic operations.

Task addition lets you combine capabilities from multiple fine-tuned models into one. Given task vectors $\tau_1, \tau_2, \ldots, \tau_k$ from $k$ different tasks, you construct a merged model by:

where:

• $\lambda$: a scaling coefficient that controls the overall magnitude of the task vector sum, typically tuned on a small validation set, often around 0.3 to 0.5
• $\tau_i$: the task vector for the $i$-th fine-tuned model
• $\theta_{\text{pretrained}}$: the frozen pretrained base model weights

The scaling coefficient $\lambda$ is important because stacking multiple task vectors can push the model far from the pretrained point. Too large a sum may degrade general capabilities; the scaling factor keeps the merged weights in a region where the pretrained model's priors still hold. Think of $\lambda$ as a dial that controls how much the merged model commits to the new tasks versus how much it retains from the general pretrained capabilities. In the original paper, $\lambda = 0.5$ worked well across a range of experiments, but the optimal value varies with the number of tasks and their degree of overlap.

Task negation is an equally interesting operation. By subtracting a task vector from the pretrained model, you can approximately remove a capability:

If a task vector captures "how to produce toxic output," subtracting it nudges the model away from that behavior without requiring additional training data or fine-tuning. The mechanism is not perfectly precise: you are subtracting the entire change that fine-tuning made, which includes both the targeted behavior and any correlated changes. But empirically, negation has shown promise for capability removal and for detoxification. It represents a form of post-hoc editing that is dramatically cheaper than retraining.

Task analogy follows the same spirit as word vector arithmetic. Just as word embeddings allow the operation king minus man plus woman to approximate queen, we can construct analogical operations in model space. Take a model fine-tuned on French sentiment analysis, subtract the French language translation task vector, and add the Spanish language translation task vector. The result might approximate a model that performs sentiment analysis in Spanish, without ever collecting Spanish sentiment training data. These analogies do not always succeed, and their reliability depends on how semantically similar the task vectors are, but they have shown empirical promise for low-resource task transfer.

The success of task arithmetic reflects a deeper regularity in how large pretrained models encode knowledge. Pretrained language models develop internal representations that are modular and reusable across tasks. When you fine-tune on task $A$, the fine-tuning process adjusts a relatively concentrated subset of parameters substantially; most parameters change only slightly. When you fine-tune on task $B$, a different subset shifts. Because many tasks use different "slots" in the model's representational space, their task vectors have limited interference when summed.

This modularity is not guaranteed by architecture; it emerges from training. Models trained on sufficiently diverse data develop the capacity to separately encode different capabilities, and fine-tuning exploits this structure by updating only the relevant modules while leaving others approximately unchanged. The task vector essentially identifies which modules a task relies on and how they should be adjusted.

This also explains when task arithmetic fails. Tasks that require conflicting parameter adjustments will interfere destructively when added. If both fine-tunes heavily modify the same layers and those modifications are in opposing directions, the sum will be noisy and the merged model will struggle on both tasks. The later methods in this chapter, TIES and DARE, address this interference problem systematically.

The idea of task vectors connects to a broader thread in machine learning research about the geometry of parameter space. Prior work on linear mode connectivity (Frankle et al., 2020) showed that models fine-tuned from the same initialization are connected by nearly loss-flat linear paths. Work on model editing (Mitchell et al., 2021) showed that specific factual associations in language models can be located and modified through targeted weight updates. Task arithmetic unified and extended these observations by showing that the difference between pretrained and fine-tuned weights is a semantically meaningful object that can be manipulated independently of the absolute weights.

The success of task arithmetic also connects to the lottery ticket hypothesis (Frankle and Carlin, 2019), which suggested that neural networks contain sparse subnetworks that do most of the work. Task vectors implicitly identify which parameters matter for a given task, and operations like DARE (discussed later) exploit this by pruning most of the task vector and keeping only the most important parameters.

Uniform task arithmetic adds all task vectors together without accounting for conflicts. When two task vectors push the same parameter in opposite directions, the result can be worse than either individual model. TIES merging (Trim, Elect Sign, and Disjoint Merge) introduces three structured steps to handle these conflicts systematically, producing merged models that are more stable and reliable than naive task addition.

The method was introduced by Yadav et al. in their 2023 paper "Resolving Interference When Merging Models." The paper's central insight is that interference between task vectors is the primary cause of performance degradation in multi-task merges, and that this interference can be substantially reduced through a principled three-step process applied before the final combination.

To understand why TIES is necessary, consider what happens when you add two task vectors that conflict. Suppose model $A$ was fine-tuned to be more formal and model $B$ was fine-tuned to be more casual. For some parameter $j$ controlling the formality dimension of the model's output, $\tau_A[j] > 0$ (push toward formal) and $\tau_B[j] < 0$ (push toward casual). Simple addition gives $\tau_A[j] + \tau_B[j]$, a value that may be small in magnitude because the two large signals cancelled. The merged model ends up with a parameter that was pushed hard in opposite directions and lands somewhere that neither task wanted: not formal, not casual, but incoherently in between.

More concretely, consider a single parameter $\theta_j$ in the pretrained model. After fine-tuning on task $A$, it shifts by $\delta_j^A$, and after fine-tuning on task $B$, by $\delta_j^B$. If $\delta_j^A > 0$ and $\delta_j^B < 0$, simple addition gives $\delta_j^A + \delta_j^B$, a value reduced in magnitude that incorporates conflicting signals. The interference is destructive: you lose the signal from both tasks without gaining a sensible compromise.

TIES addresses this through a structured three-step pipeline applied to the task vectors before combination.

Many parameters in a task vector have very small magnitudes, representing fine-tuning noise rather than meaningful task-specific adaptation. These small values accumulate from gradient descent finding locally optimal adjustments that happen to be numerically nonzero but carry little semantic signal. When you sum multiple task vectors, these small noisy values add up and can overwhelm the large, meaningful signals that encode task-specific behavior.

The trim step removes this noise by keeping only the top-$k\%$ of parameters by absolute magnitude and setting the rest to zero:

where:

• $\hat{\tau}_i[j]$: the trimmed task vector value at position $j$ for model $i$
• $\tau_i[j]$: the original task vector value at position $j$ for model $i$
• $\gamma_i^{(k)}$: the $k$-th percentile threshold of $|\tau_i|$, the magnitude threshold computed separately for each model

A typical setting keeps the top 20% of values (i.e., $k = 20$), discarding the bottom 80% as noise. This sparse representation removes clutter before the merge. The intuition is that the parameters with the largest magnitude changes are the ones the optimizer found most useful for the task. The parameters with small magnitude changes were essentially unchanged by fine-tuning and contribute more noise than signal when merged.

The trim step turns each dense task vector into a sparse one: 80% zeros and 20% nonzero values. This sparsity is what allows the next step to reason clearly about conflicts.

After trimming, each parameter position may have nonzero values from multiple task vectors, potentially with opposing signs. For a parameter that two tasks disagree about, you need a principled way to decide which direction to trust. The elect sign step resolves this by choosing a "majority vote" direction for each parameter, weighted by magnitude.

For each parameter position $j$, compute the total positive and negative momentum across all task vectors:

where:

• $m^+[j]$: total magnitude of positive updates at position $j$ across all models
• $m^-[j]$: total magnitude of negative updates at position $j$ across all models
• $\mathbb{1}[\cdot]$: indicator function (1 if condition holds, 0 otherwise)

The elected sign for position $j$ is:

where ties can be broken arbitrarily, often with a small positive bias. This elected sign vector $\gamma$ represents the consensus direction: the direction that the majority of task vectors (weighted by magnitude) want to push each parameter.

The magnitude weighting is important. A single model with a large, confident update in one direction should outweigh several models with small, uncertain updates in the opposing direction. Simply counting votes without weighting by magnitude would give equal voice to a highly confident update and a near-zero noisy one. The weighted vote ensures that the consensus direction reflects where the most important adaptations agree.

With elected signs in hand, the final merge discards any task vector contributions that conflict with the consensus:

where:

• $\mathcal{A}[j] = \{i : \hat{\tau}_i[j] \neq 0 \text{ and } \text{sign}(\hat{\tau}_i[j]) = \gamma[j]\}$: the set of models with nonzero values agreeing with the elected sign at position $j$
• $|\mathcal{A}[j]|$: the number of models in agreement at position $j$

Only task vectors with the consensus sign contribute to the average at each position. Models that disagree are silenced for that parameter. The average is taken only over the agreeing models, not over all $k$ models, so disagreeing models do not dilute the signal by contributing zeros to the denominator.

The final merged model is:

This three-step process is more conservative than simple task arithmetic. By filtering small values and resolving sign conflicts before averaging, TIES produces merged models with less destructive interference, particularly when merging three or more models with overlapping parameter sensitivities.

The three steps address distinct sources of noise in a principled sequence. Trimming removes small, incidental changes that fine-tuning made but that carry little semantic meaning. Sign election identifies the consensus direction for parameters that multiple tasks care about. Disjoint merge combines only the aligned contributions, avoiding the cancellation that would occur if opposing signals were averaged together.

Consider a worked example with three models merging a parameter that controls formality of output. Model A pushes it strongly positive (formal). Model B pushes it moderately positive. Model C pushes it weakly negative. After trimming, suppose model C's small negative value is discarded as noise. Sign election sees two positive votes and zero negative votes: the elected sign is positive. Disjoint merge averages model A's and model B's contributions. The result is a clearly positive adaptation toward formality, which is what two of the three models confidently wanted.

TIES improves over naive task arithmetic by handling sign conflicts, but it still includes all parameters that survive the trim step. DARE (Drop And Rescale) takes a different approach: randomly drop a large fraction of the task vector and rescale the remaining values to compensate. The name of the paper introducing DARE captures its spirit well: "Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch."

The method was introduced by Yu et al. in 2023. The paper's title is a reference to the video game mechanic where Mario absorbs power-ups without losing his prior abilities, which is an analogy for how model merging lets a base model absorb fine-tuned capabilities without forgetting its pretrained knowledge.

DARE is motivated by an empirical observation: fine-tuned models differ from their pretrained base in many small ways, but most of these differences are redundant. Many parameters can be set back to their pretrained values without noticeably harming the fine-tuned model's performance, as long as the remaining changed parameters are appropriately rescaled.

This observation is not obvious. You might expect that every nonzero entry in a task vector contributes something important, and that removing any of them would hurt performance. The empirical reality is different. Fine-tuning with gradient descent produces many small, correlated updates that collectively shift the model's behavior, but any individual small update contributes marginally. Setting 90% of the task vector entries back to zero while rescaling the remaining 10% preserves the model's task performance surprisingly well. The redundancy of fine-tuning updates is a property of the overparameterized regime in which modern language models operate.

Formally, DARE applies a random pruning mask $m_i \in \{0, 1\}^d$ to each task vector, where each entry is independently set to 0 with probability $p$ (the drop rate) and to 1 with probability $1 - p$:

where:

• $\hat{\tau}_i$: the pruned task vector after applying the mask
• $m_i$: the binary random mask, each entry 1 with probability $1 - p$
• $\odot$: element-wise multiplication
• $p$: the drop rate, typically set to 0.9 or higher, meaning 90% or more of entries are dropped

The key difference from TIES's trim step is that DARE uses random sampling rather than magnitude-based selection. Instead of always keeping the top-magnitude entries, DARE draws a random mask and keeps whichever entries the mask selects. This randomness prevents the merge from systematically ignoring any particular parameter, which can be beneficial when the large-magnitude entries of different task vectors conflict with each other.

Dropping 90% of task vector entries would normally collapse the scale of the updates and effectively undo most of the fine-tuning. DARE compensates by rescaling the surviving values by $\frac{1}{1-p}$:

where:

• $\tilde{\tau}_i$: the rescaled, pruned task vector
• $\frac{1}{1-p}$: the rescaling factor that preserves expected magnitude, analogous to the inverse dropout scaling used in neural network training

This is mathematically identical to the dropout trick used during neural network training, where activations are randomly zeroed and then scaled up by the inverse survival probability to preserve expected values. In expectation, the rescaled pruned vector has the same mean as the original:

So in expectation, DARE's pruned and rescaled task vector is unbiased relative to the original. Each individual realization will differ from the original because of the randomness, but on average, the vector preserves the full magnitude of the original task updates. This is what makes DARE safe to apply aggressively: you are not systematically reducing the scale of updates, only introducing sparsity while preserving the expected scale.

DARE is not itself a merging method but rather a preprocessing step applied to each task vector before merging. After applying DARE to each task vector independently, you can combine the sparsified vectors using any merging strategy, such as simple averaging or TIES.

The full DARE + TIES pipeline works as follows. First, compute task vectors $\tau_i = \theta_i - \theta_{\text{pretrained}}$ for each fine-tuned model. Second, apply DARE to each task vector: randomly drop $p$ fraction of entries and rescale the survivors by $\frac{1}{1-p}$. Third, apply TIES to the resulting sparse task vectors: trim any residual small values, elect signs by magnitude-weighted majority vote, and disjoint merge. Finally, add the merged task vector to the pretrained model: $\theta_{\text{merged}} = \theta_{\text{pretrained}} + \lambda \cdot \tau_{\text{merged}}$.

The combination often outperforms either DARE or TIES alone because the two methods address complementary sources of noise. DARE's random sparsification reduces redundancy by independently sparsifying each task vector, preventing the same small, correlated updates from accumulating across models. TIES's sign election then resolves any remaining conflicts between the surviving nonzero entries. The two steps together produce a cleaner merged task vector with less noise and more reliable signal.

The fact that dropping 90% of task vector entries still preserves most model behavior is initially surprising. Why should such aggressive pruning work?

The explanation connects to overparameterization. Modern language models have far more parameters than strictly necessary to represent any single task. When fine-tuning adjusts many parameters, only a fraction of them are truly essential to the task's performance. The majority of nonzero entries in the task vector represent small, redundant adjustments that collectively push the model in the right direction but individually contribute little. DARE acts as a noise filter, discarding many of the small, incidental changes that fine-tuning makes and retaining only the most important structural modifications.

This finding aligns with the observations from sparse fine-tuning literature, where methods like LoRA demonstrate that you can fine-tune a large model using only a small number of low-rank updates and achieve performance comparable to full fine-tuning. The effective rank of the fine-tuning update is much lower than the full parameter count, which means most of the task vector lives in a low-dimensional subspace. DARE exploits this by randomly sampling from the full parameter space, statistically covering the important subspace while discarding much of the noise.

The randomness in DARE also serves a practical purpose for merging multiple models. When two task vectors have conflicting values at the same parameter position, DARE randomly zeros out one or both of them with high probability. This stochastic conflict resolution is less systematic than TIES's sign election but is computationally cheaper and can reduce interference in ways that deterministic methods might miss.

Let's implement these four merging methods from scratch using PyTorch. We will work with small toy models to keep the computation tractable, but the same code logic scales directly to full-size language models. The key operations, computing task vectors, applying masks, electing signs, and combining vectors, are all element-wise or reduction operations over flat parameter vectors that work identically regardless of model size.

We start by importing the necessary libraries and defining a simple linear model architecture that we can fine-tune on toy tasks. Using a small model lets us inspect the behavior of each method clearly without waiting for large-scale training.

We define a small feedforward network and two simple regression tasks that require different parameter configurations. Each task is defined by a random projection of the input, so the two tasks require different linear functions of the same input space. A model that perfectly solves task A will not generally solve task B, making this a concrete test of each merging method's ability to combine capabilities.

A simple training loop fine-tunes the pretrained model on each task and returns the resulting fine-tuned model. The key point is that both fine-tunes start from the same pretrained weights, which is the essential condition for model merging to work well.

All merging operations work on flat vectors of parameters. Converting model parameters to a flat vector is the first step in any merging pipeline. We define helpers to convert between flat vectors and model instances.

The flat vector representation makes the merging code clean and general: all the operations work on 1D tensors regardless of the specific architecture. When scaling to real models, you would typically work with state dictionaries and apply operations layer by layer to avoid loading all parameters into memory simultaneously, but the logic is identical.

The simplest merge: compute the element-wise mean of all fine-tuned parameter vectors.

Task arithmetic computes task vectors, scales them, and adds them back to the pretrained model. The scaling coefficient $\lambda$ is the key hyperparameter controlling how much of the fine-tuned direction to incorporate.

The TIES implementation follows the three steps: trim by magnitude percentile, elect sign by weighted majority vote, and disjoint merge using only agreeing entries.

Notice that agree_count.clamp(min=1.0) prevents division by zero in cases where no model has a nonzero value at some position after trimming. This can happen if all task vectors had small values at a particular position that were all trimmed away.

DARE applies random masking with rescaling to each task vector independently, then combines with simple averaging.

The rescaling by 1.0 - drop_rate ensures that the surviving entries carry the same expected magnitude as the original task vector, compensating for the entries that were dropped.

We evaluate each merged model on both tasks and compare their losses. This gives a concrete picture of how each method balances performance across multiple tasks.

The results illustrate the core trade-off in model merging. Individual fine-tuned models excel on their own task but perform close to the pretrained baseline on the other task: they have specialized at the expense of generality. Each merging strategy balances performance across both tasks differently, with more sophisticated methods generally achieving better combined performance by reducing interference between task-specific updates. The pretrained model is a lower bound on merge quality: any merge that performs worse than the pretrained baseline on a task has failed its purpose.

The magnitude distribution of task vector entries reveals the core property that makes DARE and TIES effective: most fine-tuning changes are small and concentrated near zero, while only a minority of parameters receive large updates.

The distribution you see here is typical of fine-tuned models at any scale. The spike near zero represents parameters that fine-tuning barely touched. The long tail represents the relatively small number of parameters that were significantly adjusted to accommodate the task. Trimming in TIES targets this structure explicitly: it keeps the tail and discards the spike. DARE samples randomly from both regions, but because the spike contains far more parameters, most of what it keeps are parameters from the tail when the drop rate is high enough.

The scaling coefficient $\lambda$ controls how aggressively task vectors are applied. Too small, and the merged model barely departs from the pretrained baseline. Too large, and the combined updates push the model into territory where neither task can perform well. Scanning across lambda values reveals the operating range and the optimal point for each method.

The curves reveal that both Task Arithmetic and TIES initially improve as lambda grows from zero, with performance gains accelerating as the task vectors contribute more of their signal. However, at some threshold, further increases in lambda cause degradation because the combined updates push the model beyond the region where it performs well on both tasks simultaneously. TIES tends to be more stable at higher lambda values because sign conflicts have been resolved before the merge, reducing the chance that a large lambda amplifies destructive interference.

The practical takeaway is that lambda requires tuning. The optimal value depends on the number of tasks, the degree of overlap between task vectors, and the scale of the task vector magnitudes. A small validation set covering the tasks you care about is usually sufficient to find a good lambda with a simple grid search.

The plot confirms that performance is relatively stable across a wide range of drop rates before degrading sharply as the drop rate approaches 1.0. This plateau demonstrates why DARE can aggressively prune task vectors: most entries are redundant, and the rescaling step ensures the surviving entries carry the same expected signal as the full task vector. In practice, drop rates of 0.9 are standard, and rates as high as 0.95 often work well, particularly when combining DARE with TIES.

Model merging is a powerful idea, but it comes with important limitations that constrain where and how it can be applied. Understanding these limitations is essential for using the methods effectively and for recognizing when to reach for a different approach.

The most fundamental requirement is architectural compatibility. All models being merged must share exactly the same architecture: the same number of layers, the same layer dimensions, and the same parameter layout. You cannot merge a 7B model with a 13B model, or a model using rotary position embeddings with one using absolute position embeddings. In practice, this restricts merging to models derived from the same pretrained base, which is the dominant use case on platforms like Hugging Face where many practitioners fine-tune and share variants of Llama, Mistral, or Phi. The Hugging Face Hub hosts thousands of fine-tuned variants of popular base models, and model merging has become a standard technique for combining these variants into capable generalists.

Even when architectures match, semantic alignment of parameters is not guaranteed. Weight averaging and task arithmetic implicitly assume that the $j$-th parameter in model $A$ encodes the same abstract concept as the $j$-th parameter in model $B$. For models derived from the same pretrained checkpoint, this assumption generally holds because fine-tuning makes relatively small perturbations and the parameter indices retain their semantic roles from pretraining. For models trained from different initializations, even with the same architecture, the same index may encode completely unrelated things and averaging them is meaningless. This is why techniques like SLERP (spherical linear interpolation, which accounts for the curved geometry of weight space) and model permutation alignment (which finds permutations of hidden units that improve alignment before merging) are active research areas, addressing the harder case of merging models that were not trained from the same starting point.

Hyperparameter sensitivity is another practical challenge. The scaling coefficient $\lambda$ in task arithmetic and TIES, the trim percentage in TIES, and the drop rate in DARE all affect merge quality substantially. Finding good values requires either a validation set covering all tasks or grid search over the hyperparameter space. Automated merge optimization tools, including tools that treat the merge coefficients as learnable parameters and optimize them with a small amount of gradient descent on a validation set, have emerged to address this problem. But the field lacks reliable, transferable heuristics that work across models and tasks without tuning, which means practitioners typically need to experiment.

Model merging also does not solve the problem of fundamentally conflicting tasks. If two tasks require opposing model behaviors, their task vectors will be in direct conflict for many parameters. TIES's sign election will pick one direction and discard the other, effectively forcing the merged model to choose a winner rather than combining the capabilities. True multi-task capability in such cases requires either specialized training, a mixture-of-experts architecture where different experts handle different tasks, or task-routing mechanisms that select different parameter sets for different inputs. Model merging works best when the tasks being merged are complementary or at least orthogonal in their parameter usage, not when they pull in directly opposite directions.

There is also the question of measuring success. A merged model might perform better on average across tasks but worse on any individual task compared to the specialized fine-tune. Whether this is acceptable depends on the use case. If you need a model that handles ten different tasks reasonably well, an average improvement might be exactly what you want. If you need peak performance on one specific task, the specialized fine-tune is likely better. The benchmarks used to evaluate merged models matter, and cherry-picked benchmarks can make any method look better than it is.

Despite these limitations, model merging has had real practical impact. The Hugging Face Open LLM Leaderboard has consistently featured merged models near the top, with practitioners combining specialized fine-tunes in creative ways to outperform any individual component. Tools like mergekit have democratized the process, making it possible for practitioners without training infrastructure to produce competitive models simply by merging publicly available fine-tunes. mergekit supports all of the methods covered in this chapter, as well as more recent techniques like SLERP and linear interpolation with per-layer merge coefficients, and it handles the practical details of working with large model files without loading all parameters into memory simultaneously.

The emerging field of "model merging as capability composition" continues to push these methods forward, with newer techniques addressing gradient-based optimization of merge coefficients, layer-wise merging strategies where different layers use different merge settings, and combinations with quantization that produce merged models small enough to deploy on consumer hardware. The key observation that drives this entire field is that fine-tuned variants of the same base model form a rich combinatorial space of capabilities, and merging is the zero-cost way to explore that space.

When you want to apply model merging in practice, a few principles guide the choice of method and hyperparameters.

Start with the simplest method that might work. Weight averaging is the right baseline when you are merging checkpoints from different stages of a single training run or multiple fine-tunes with very similar objectives. Task arithmetic is the right starting point for merging models fine-tuned on distinct tasks, because the task vector formulation makes the operation interpretable and the scaling coefficient provides a straightforward knob.

Move to TIES when you are merging three or more models and notice that task arithmetic produces inconsistent results or that some tasks perform much worse in the merge than they do individually. TIES's sign election is most valuable when the models being merged have overlapping parameter sensitivities with opposing directions, which is common when fine-tuning on tasks that require different output styles or factual domains.

Consider DARE as a preprocessing step when you have reason to believe that your task vectors contain substantial noise, for instance when fine-tuning was performed with a high learning rate or for many epochs, or when the fine-tuned models show signs of overfitting. DARE's aggressive sparsification tends to reduce the footprint of each task vector, making the subsequent merge cleaner.

For the scaling coefficient $\lambda$, a sensible default is 0.4 when merging two models and 0.3 when merging three or more. These values are consistent with what practitioners report on the Hugging Face community forums and with the original paper recommendations. Always validate on a held-out set before deploying a merged model.

Model merging combines the weights of multiple models into a single model without additional training, enabling capability composition at near-zero cost. The methods covered in this chapter form a progression of increasing sophistication, each addressing a specific limitation of the previous approach.

Weight averaging computes the element-wise mean of model parameters. It works best when models are fine-tuned from the same pretrained checkpoint because they remain in the same region of weight space, making sure that the average sits in a low-loss region that satisfies all objectives reasonably well. The model soup variant improves on uniform averaging by greedily selecting which models to include based on validation performance.

Task arithmetic defines task vectors as the difference between fine-tuned and pretrained weights, shifting the perspective from absolute weights to relative changes. These vectors can be scaled, added, subtracted, and composed algebraically, enabling capability transfer and removal without retraining. The scaling coefficient $\lambda$ controls how aggressively task vectors are applied and must be tuned to balance task performance with general capability retention.

TIES merging addresses parameter conflicts through three steps: trim small-magnitude values from each task vector to remove noise, elect a consensus sign direction for each parameter position via magnitude-weighted voting, and merge only the values that agree with the elected sign, discarding conflicting contributions. This structured conflict resolution produces more stable merged models, especially when combining three or more specialized fine-tunes.

DARE merging randomly drops a large fraction of task vector entries, typically 90%, and rescales the survivors to preserve expected magnitude. This aggressive sparsification reduces redundancy and interference in a stochastic way and works best as a preprocessing step combined with TIES for the subsequent merge. The empirical finding that high drop rates preserve model performance reflects the overparameterized nature of modern language models, where most fine-tuning updates are redundant.

All methods require models to share the same architecture and work best when models are derived from the same pretrained base. The scaling coefficient $\lambda$ controls how aggressively task vectors are applied and must be tuned per use case. Tools like mergekit make these methods practical for full-scale language models without requiring specialized infrastructure, and the open-source community has demonstrated that merged models can achieve competitive benchmark performance while combining capabilities from multiple specialized fine-tunes.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about model merging techniques.