Model Merging Applications › Multi-Task Merging

Multi-task merging is the most direct application of the techniques you learned in the previous chapter. The goal is to produce a single model that performs well across several tasks without training on combined data. Instead of learning a multitask objective jointly, you merge models that were each fine-tuned independently on a single task.

Multi-task training requires simultaneous access to all task datasets, careful loss balancing across tasks, and significant compute. The loss balancing problem alone is non-trivial: if one task has a much larger dataset than another, a naive joint objective will overfit to the dominant task while starving the minority tasks of gradient signal. Researchers have developed sophisticated sampling strategies and loss weighting schemes to address this, but tuning them requires its own round of experimentation. Merging sidesteps these constraints entirely. You can assemble a multi-task model after the fact from already-deployed checkpoints, combine models trained by different teams on proprietary data, and iterate quickly without re-training.

Another underappreciated advantage of merging over multi-task training is that fine-tuning each model independently allows you to use the optimal hyperparameters for each task. A task requiring many fine-tuning steps for good performance will not interfere with a task that converges quickly. The fine-tunes can use different learning rates, batch sizes, and even different data augmentations. Merging then combines these optimally-tuned specialists into a single model.

Consider a concrete scenario: a company operates three internal applications that each need an LLM. One tool handles customer support email classification, requiring a fine-tune on labeled support tickets. Another tool generates code completions for an internal scripting language, fine-tuned on internal code repositories. A third tool produces structured JSON summaries from long documents, fine-tuned on document-summary pairs. Training a single multitask model from scratch would require collecting all three datasets in one place, a process that may violate data governance rules if different teams own different datasets. Merging lets each team independently fine-tune their model and then combine the results without sharing raw data.

The tradeoff is that merging cannot exceed the performance ceiling set by the individual fine-tunes. Multi-task training, when done well with sufficient data for all tasks, typically beats merging. But merging is often good enough, and the engineering simplicity has practical value. In practice, multi-task merging is most effective when the individual tasks are related enough to share useful representations but different enough that they would require careful balancing in joint training.

Recall from the previous chapter that a task vector $\tau_t$ is the difference between a fine-tuned model and the base model:

$$\tau_t = \theta_t - \theta_{\text{base}}$$

where:

• $\theta_t$: the weight vector of a model fine-tuned on task $t$
• $\theta_{\text{base}}$: the shared pre-trained base model weights
• $\tau_t$: the task-specific "delta" encoding what was learned during fine-tuning

This representation is powerful because it separates the task-specific learned information from the general-purpose language capabilities already present in the base model. The base model already knows grammar, world facts, and reasoning patterns. The task vector contains only the incremental adjustments needed to specialize for a particular task.

When you merge two task vectors by addition, the merged model is:

$$\theta_{\text{merged}} = \theta_{\text{base}} + \lambda_1 \tau_1 + \lambda_2 \tau_2$$

where $\lambda_1$ and $\lambda_2$ are scaling coefficients. The core challenge is interference: some parameters in $\tau_1$ and $\tau_2$ will point in conflicting directions, and simply adding them causes degradation on both tasks.

To understand interference intuitively, imagine two people editing the same document simultaneously. If one person wants to change a word from "happy" to "joyful" and the other wants to change it from "happy" to "content," the result of combining both edits might be garbled text. In weight space, the analogous problem occurs when two fine-tunes try to push the same parameter in opposite directions. The sum of the two opposing changes partially cancels, leaving the parameter somewhere between the two desired values, which may serve neither task well.

The severity of interference depends on the structural relationship between tasks. Tasks that share linguistic structure but differ in output format (e.g., classification versus generation) tend to have low interference because they modify different components of the network. Tasks that compete for the same internal representations (e.g., two classification tasks that use similar reasoning patterns but require different final mappings) tend to have higher interference.

Before merging, it is useful to quantify expected interference. One practical metric is the cosine similarity between task vectors:

$$\text{sim}(\tau_1, \tau_2) = \frac{\tau_1 \cdot \tau_2}{\|\tau_1\| \cdot \|\tau_2\|}$$

High positive similarity means the two fine-tunes modified weights in roughly the same direction, and merging is likely to be clean. High negative similarity signals heavy interference, and you may need TIES or DARE to handle the conflict. Near-zero similarity suggests the tasks occupy nearly orthogonal parts of parameter space, which often merges well.

The cosine similarity is a global summary statistic computed across all parameters simultaneously. A more granular analysis computes the similarity layer by layer, which reveals whether interference is concentrated in specific parts of the network. It is common to find, for example, that two task vectors are nearly orthogonal in the first twelve layers but show substantial conflict in the final four layers near the output head. This diagnostic guides selective merging strategies where you apply different algorithms to different depth ranges.

A complementary diagnostic is the overlap fraction: the fraction of parameters where both task vectors have non-negligible magnitude. If $\tau_1$ has large values in parameters indexed by set $S_1$ and $\tau_2$ has large values in parameters indexed by set $S_2$, the overlap is $|S_1 \cap S_2| / |S_1 \cup S_2|$. Low overlap predicts clean merging because the tasks edit mostly non-overlapping weights. High overlap combined with opposing signs is the worst case for interference.

The standard workflow for multi-task merging involves five steps.

First, verify that all fine-tuned models share the same base architecture and tokenizer. Models fine-tuned from different checkpoints of the same base (e.g., a 10k-step and 100k-step checkpoint) may still merge, but divergence in weight scale can cause numerical problems. If you are unsure about compatibility, load all models and compare parameter counts, layer names, and weight magnitudes.

Second, compute task vectors for each model and inspect their norms and pairwise cosine similarities. This gives you an early signal about which task pairs will merge cleanly. Save the task vectors as separate tensors rather than recomputing them repeatedly during the sweep, because loading multiple full model checkpoints simultaneously can be memory-intensive.

Third, choose a merging algorithm. For two or three tasks with positive pairwise similarity, simple task arithmetic with equal weights often works. For more tasks or conflicting vectors, use TIES to resolve sign conflicts before summing, or DARE to sparsify vectors and reduce interference. When in doubt, try both and compare on validation data.

Fourth, sweep the scaling coefficient $\lambda$ (and TIES/DARE hyperparameters) using a small held-out validation set from each task. The optimal $\lambda$ is rarely 1.0 for multi-task merges; values between 0.3 and 0.8 are typical. For merges involving three or more tasks, you may need to tune per-task coefficients independently, which turns the search into a small hyperparameter optimization problem.

Fifth, evaluate the merged model against the individual fine-tunes to measure the "task loss," defined as:

$$\text{task loss}_t = \frac{\text{perf}(\theta_t) - \text{perf}(\theta_{\text{merged}})}{\text{perf}(\theta_t)}$$

A task loss below 5% is generally considered acceptable for a multi-task merge. This means the merged model performs at least 95% as well as the dedicated fine-tune on each individual task, while adding a second capability with no additional memory cost.

The two-task case generalizes to $n$ tasks straightforwardly by extending the sum:

$$\theta_{\text{merged}} = \theta_{\text{base}} + \sum_{t=1}^{n} \lambda_t \tau_t$$

In practice, the quality of the merge degrades as $n$ increases. With two tasks and low interference, you can achieve a combined normalized score above 0.9 for both tasks. With eight tasks, achieving a score above 0.7 on all eight simultaneously is already challenging, because even small pairwise interference effects accumulate across many task vectors. TIES and DARE become more important as $n$ grows precisely because they reduce the effective magnitude of each task vector before summing, limiting the total interference budget.

One strategy for scaling to many tasks is hierarchical merging: merge subsets of closely related tasks first (creating intermediate "domain expert" models), then merge the domain experts into a final generalist. For example, you might first merge a sentiment fine-tune with an NLI fine-tune (both natural language understanding tasks with positive pairwise similarity), then merge the resulting NLU expert with a code generation fine-tune (which occupies largely orthogonal parameter space). This two-stage process reduces the interference at each step compared to a direct $n$-way merge.