Model Merging › Summary

Model merging combines the weights of multiple models into a single model without additional training, enabling capability composition at near-zero cost. The methods covered in this chapter form a progression of increasing sophistication, each addressing a specific limitation of the previous approach.

Weight averaging computes the element-wise mean of model parameters. It works best when models are fine-tuned from the same pretrained checkpoint because they remain in the same region of weight space, making sure that the average sits in a low-loss region that satisfies all objectives reasonably well. The model soup variant improves on uniform averaging by greedily selecting which models to include based on validation performance.

Task arithmetic defines task vectors as the difference between fine-tuned and pretrained weights, shifting the perspective from absolute weights to relative changes. These vectors can be scaled, added, subtracted, and composed algebraically, enabling capability transfer and removal without retraining. The scaling coefficient $\lambda$ controls how aggressively task vectors are applied and must be tuned to balance task performance with general capability retention.

TIES merging addresses parameter conflicts through three steps: trim small-magnitude values from each task vector to remove noise, elect a consensus sign direction for each parameter position via magnitude-weighted voting, and merge only the values that agree with the elected sign, discarding conflicting contributions. This structured conflict resolution produces more stable merged models, especially when combining three or more specialized fine-tunes.

DARE merging randomly drops a large fraction of task vector entries, typically 90%, and rescales the survivors to preserve expected magnitude. This aggressive sparsification reduces redundancy and interference in a stochastic way and works best as a preprocessing step combined with TIES for the subsequent merge. The empirical finding that high drop rates preserve model performance reflects the overparameterized nature of modern language models, where most fine-tuning updates are redundant.

All methods require models to share the same architecture and work best when models are derived from the same pretrained base. The scaling coefficient $\lambda$ controls how aggressively task vectors are applied and must be tuned per use case. Tools like mergekit make these methods practical for full-scale language models without requiring specialized infrastructure, and the open-source community has demonstrated that merged models can achieve competitive benchmark performance while combining capabilities from multiple specialized fine-tunes.