Model Merging Applications: Multi-Task, Style

The previous chapter on Model Merging introduced the core algorithms: weight averaging, task arithmetic, TIES merging, and DARE. Those techniques answer the question of how to combine two or more models mathematically. This chapter addresses a more practical question: what can you build with model merging, and how do you know if your merged model is any good?

The answer turns out to be remarkably broad. Researchers and practitioners have applied merging to combine models trained on different tasks, blend stylistic characteristics from specialized fine-tunes, compose capabilities that would be expensive or impossible to train from scratch, and create generalist models without touching training data. Each of these use cases requires a merging algorithm, plus careful thinking about model compatibility, parameter selection, and evaluation strategy.

The appeal of merging becomes clearest when you consider the alternative. Building a single model that handles sentiment analysis, summarization, question answering, and code generation traditionally requires either (a) training a massive multitask model from scratch with all four datasets simultaneously, or (b) maintaining four separate deployed models, each occupying GPU memory and requiring separate inference infrastructure. Merging offers a third path: take four independently fine-tuned specialists, combine them into a single set of weights, and deploy one model that handles all four tasks. The cost of the merge operation itself is trivial, measured in seconds on a single GPU.

This chapter walks through the four main application domains one by one, with worked examples and code you can adapt to your own models. We finish with a practical guide to merging evaluation, which is often the bottleneck in real projects.

Multi-task merging is the most direct application of the techniques you learned in the previous chapter. The goal is to produce a single model that performs well across several tasks without training on combined data. Instead of learning a multitask objective jointly, you merge models that were each fine-tuned independently on a single task.

Multi-task training requires simultaneous access to all task datasets, careful loss balancing across tasks, and significant compute. The loss balancing problem alone is non-trivial: if one task has a much larger dataset than another, a naive joint objective will overfit to the dominant task while starving the minority tasks of gradient signal. Researchers have developed sophisticated sampling strategies and loss weighting schemes to address this, but tuning them requires its own round of experimentation. Merging sidesteps these constraints entirely. You can assemble a multi-task model after the fact from already-deployed checkpoints, combine models trained by different teams on proprietary data, and iterate quickly without re-training.

Another underappreciated advantage of merging over multi-task training is that fine-tuning each model independently allows you to use the optimal hyperparameters for each task. A task requiring many fine-tuning steps for good performance will not interfere with a task that converges quickly. The fine-tunes can use different learning rates, batch sizes, and even different data augmentations. Merging then combines these optimally-tuned specialists into a single model.

Consider a concrete scenario: a company operates three internal applications that each need an LLM. One tool handles customer support email classification, requiring a fine-tune on labeled support tickets. Another tool generates code completions for an internal scripting language, fine-tuned on internal code repositories. A third tool produces structured JSON summaries from long documents, fine-tuned on document-summary pairs. Training a single multitask model from scratch would require collecting all three datasets in one place, a process that may violate data governance rules if different teams own different datasets. Merging lets each team independently fine-tune their model and then combine the results without sharing raw data.

The tradeoff is that merging cannot exceed the performance ceiling set by the individual fine-tunes. Multi-task training, when done well with sufficient data for all tasks, typically beats merging. But merging is often good enough, and the engineering simplicity has practical value. In practice, multi-task merging is most effective when the individual tasks are related enough to share useful representations but different enough that they would require careful balancing in joint training.

Recall from the previous chapter that a task vector $\tau_t$ is the difference between a fine-tuned model and the base model:

where:

• $\theta_t$: the weight vector of a model fine-tuned on task $t$
• $\theta_{\text{base}}$: the shared pre-trained base model weights
• $\tau_t$: the task-specific "delta" encoding what was learned during fine-tuning

This representation is powerful because it separates the task-specific learned information from the general-purpose language capabilities already present in the base model. The base model already knows grammar, world facts, and reasoning patterns. The task vector contains only the incremental adjustments needed to specialize for a particular task.

When you merge two task vectors by addition, the merged model is:

where $\lambda_1$ and $\lambda_2$ are scaling coefficients. The core challenge is interference: some parameters in $\tau_1$ and $\tau_2$ will point in conflicting directions, and simply adding them causes degradation on both tasks.

To understand interference intuitively, imagine two people editing the same document simultaneously. If one person wants to change a word from "happy" to "joyful" and the other wants to change it from "happy" to "content," the result of combining both edits might be garbled text. In weight space, the analogous problem occurs when two fine-tunes try to push the same parameter in opposite directions. The sum of the two opposing changes partially cancels, leaving the parameter somewhere between the two desired values, which may serve neither task well.

The severity of interference depends on the structural relationship between tasks. Tasks that share linguistic structure but differ in output format (e.g., classification versus generation) tend to have low interference because they modify different components of the network. Tasks that compete for the same internal representations (e.g., two classification tasks that use similar reasoning patterns but require different final mappings) tend to have higher interference.

Before merging, it is useful to quantify expected interference. One practical metric is the cosine similarity between task vectors:

High positive similarity means the two fine-tunes modified weights in roughly the same direction, and merging is likely to be clean. High negative similarity signals heavy interference, and you may need TIES or DARE to handle the conflict. Near-zero similarity suggests the tasks occupy nearly orthogonal parts of parameter space, which often merges well.

The cosine similarity is a global summary statistic computed across all parameters simultaneously. A more granular analysis computes the similarity layer by layer, which reveals whether interference is concentrated in specific parts of the network. It is common to find, for example, that two task vectors are nearly orthogonal in the first twelve layers but show substantial conflict in the final four layers near the output head. This diagnostic guides selective merging strategies where you apply different algorithms to different depth ranges.

A complementary diagnostic is the overlap fraction: the fraction of parameters where both task vectors have non-negligible magnitude. If $\tau_1$ has large values in parameters indexed by set $S_1$ and $\tau_2$ has large values in parameters indexed by set $S_2$, the overlap is $|S_1 \cap S_2| / |S_1 \cup S_2|$. Low overlap predicts clean merging because the tasks edit mostly non-overlapping weights. High overlap combined with opposing signs is the worst case for interference.

The standard workflow for multi-task merging involves five steps.

First, verify that all fine-tuned models share the same base architecture and tokenizer. Models fine-tuned from different checkpoints of the same base (e.g., a 10k-step and 100k-step checkpoint) may still merge, but divergence in weight scale can cause numerical problems. If you are unsure about compatibility, load all models and compare parameter counts, layer names, and weight magnitudes.

Second, compute task vectors for each model and inspect their norms and pairwise cosine similarities. This gives you an early signal about which task pairs will merge cleanly. Save the task vectors as separate tensors rather than recomputing them repeatedly during the sweep, because loading multiple full model checkpoints simultaneously can be memory-intensive.

Third, choose a merging algorithm. For two or three tasks with positive pairwise similarity, simple task arithmetic with equal weights often works. For more tasks or conflicting vectors, use TIES to resolve sign conflicts before summing, or DARE to sparsify vectors and reduce interference. When in doubt, try both and compare on validation data.

Fourth, sweep the scaling coefficient $\lambda$ (and TIES/DARE hyperparameters) using a small held-out validation set from each task. The optimal $\lambda$ is rarely 1.0 for multi-task merges; values between 0.3 and 0.8 are typical. For merges involving three or more tasks, you may need to tune per-task coefficients independently, which turns the search into a small hyperparameter optimization problem.

Fifth, evaluate the merged model against the individual fine-tunes to measure the "task loss," defined as:

A task loss below 5% is generally considered acceptable for a multi-task merge. This means the merged model performs at least 95% as well as the dedicated fine-tune on each individual task, while adding a second capability with no additional memory cost.

The two-task case generalizes to $n$ tasks straightforwardly by extending the sum:

In practice, the quality of the merge degrades as $n$ increases. With two tasks and low interference, you can achieve a combined normalized score above 0.9 for both tasks. With eight tasks, achieving a score above 0.7 on all eight simultaneously is already challenging, because even small pairwise interference effects accumulate across many task vectors. TIES and DARE become more important as $n$ grows precisely because they reduce the effective magnitude of each task vector before summing, limiting the total interference budget.

One strategy for scaling to many tasks is hierarchical merging: merge subsets of closely related tasks first (creating intermediate "domain expert" models), then merge the domain experts into a final generalist. For example, you might first merge a sentiment fine-tune with an NLI fine-tune (both natural language understanding tasks with positive pairwise similarity), then merge the resulting NLU expert with a code generation fine-tune (which occupies largely orthogonal parameter space). This two-stage process reduces the interference at each step compared to a direct $n$-way merge.

Style merging applies model merging to a different objective: blending the stylistic or behavioral characteristics of two fine-tuned models. Rather than combining task competencies, you are combining writing styles, tones, or domain vocabularies.

Style is not a single, localized property of a model, and this makes style merging both interesting and challenging to reason about. Different components of the transformer architecture carry different aspects of what we perceive as writing style when reading the output.

Attention heads encode patterns about which tokens attend to which, shaping syntax and reference structure. Feed-forward layers encode knowledge and word associations, adding to vocabulary choice, domain-specific terminology, and the kinds of information the model is likely to include in a response. Layer normalization parameters affect scale and expressiveness. Even the output embedding matrix plays a role, because it determines the logit distribution over the vocabulary from which the final token is sampled.

When you fine-tune a model on formal business prose versus casual conversational text, the resulting task vectors will differ primarily in the feed-forward layers of the middle and later transformer blocks, where high-level semantic and stylistic patterns are stored. Early layers, which handle low-level syntax and token identity, change less. This has been confirmed empirically by comparing the per-layer magnitude of task vectors: the early-layer magnitudes are typically 3 to 5 times smaller than the late-layer magnitudes for style-focused fine-tunes.

This uneven distribution has a practical consequence: you can selectively merge style by applying different weights $\lambda$ to different layer groups, blending more heavily in the layers that carry stylistic information. This is layer-wise merging, which is distinct from applying a single global coefficient to all parameters equally.

Layer-wise merging assigns a separate coefficient to each transformer layer instead of a single global $\lambda$:

where:

• $\theta^{(l)}$: the parameters in layer $l$
• $\lambda_l$: the per-layer blending coefficient
• $\tau_1^{(l)}, \tau_2^{(l)}$: task vectors for each model at layer $l$

Notice that this formulation uses $\lambda_l$ and $(1 - \lambda_l)$, which constrains the two models to share equal total contribution at each layer. When $\lambda_l = 0.5$, each layer is an exact midpoint between the two source models' representations of that layer. When $\lambda_l = 0.8$, the layer leans heavily toward Model 1 while still incorporating some influence from Model 2.

Setting $\lambda_l = 0.5$ for all layers gives equal blending throughout the network. Setting $\lambda_l$ close to 1.0 for early layers (preserving the first model's syntax handling) and $\lambda_l = 0.5$ for later layers (blending style) is a common practical choice. A ramp profile, where $\lambda_l$ transitions from 1.0 in the first few layers to 0.5 in the later layers, has been found effective across several published experiments.

The key intuition behind layer-wise merging for style is that you want syntactic competence from one source (to ensure grammatically coherent output) while blending the high-level behavioral tendencies of both sources (to achieve the desired style mixture). Applying a single global $\lambda$ to the entire network ignores this natural separation of function between early and late layers.

Style merging has found practical application in several scenarios that illustrate its distinctive value. One prominent use case is persona blending in conversational AI. A company might have a formal customer service assistant fine-tuned on professional support interactions and a friendly chatbot fine-tuned on casual dialogue. Rather than maintaining both separately or training a new persona from scratch, a style merge between the two can produce a model that is professionally competent but conversationally approachable. The $\lambda$ coefficient becomes a dial that the product team can adjust to find the right balance between formal and casual.

Another application is domain vocabulary injection. A model fine-tuned on medical literature develops a vocabulary and reasoning style specific to clinical contexts. A model fine-tuned on patient-facing health education materials develops an accessible, simplified style. Merging them with appropriate layer-wise coefficients can produce a model that understands medical concepts at a clinical level but communicates them in patient-friendly language. This is difficult to achieve through joint fine-tuning because the two corpora pull the model in different stylistic directions simultaneously.

Style evaluation is inherently subjective, which makes it harder than task performance evaluation. Practical approaches include:

Automated style classifiers can measure the degree to which merged model outputs match the style of each source model. If you fine-tuned on formal and informal corpora, a text classifier trained on those same corpora can quantify the style blend ratio in generated outputs. A well-calibrated classifier gives you a scalar "formality score" for each generated sample, letting you track how the score distribution shifts as you vary the merge coefficient.

Perplexity on held-out examples from each style provides a proxy for whether the model has retained each style's characteristics. A well-merged style model should have lower perplexity on both style corpora than a model trained only on one. If the merged model has higher perplexity than either source model on both corpora, the merge has degraded both styles rather than blending them, which usually signals that the task vectors were interfering heavily.

Distributional metrics from the natural language generation literature can also be useful here. MAUVE, for example, measures the divergence between the distribution of generated text and a reference distribution. You can compute MAUVE separately against each source style corpus to quantify how well the merged model covers each style's distributional characteristics.

Human evaluation remains the gold standard for style, but it is expensive. A practical shortcut is to generate completions of the same prompts with the source models and the merged model, then present them side-by-side to evaluators who rate which model each output most resembles. A well-calibrated style merge should produce outputs that evaluators consistently rate as intermediate between the two source styles, rather than clearly belonging to one or the other.

Capability composition goes beyond combining task-specific fine-tunes to assembling models with capabilities not present in any single source model by combining specialized base capabilities. This is perhaps the most technically ambitious application of merging, and also the one that has attracted the most research attention in recent years.

One prominent example is combining a model fine-tuned for mathematical reasoning with a model fine-tuned on scientific domain knowledge. Neither model alone can reason about domain-specific quantitative problems: the math model lacks the domain vocabulary and conceptual framework, while the science model lacks the systematic quantitative reasoning patterns. A merge of both can potentially exhibit emergent ability to apply mathematical reasoning within the scientific domain, producing a model that can work through numerical problems in biology, chemistry, or physics with both technical accuracy and domain-appropriate framing.

This works because the two capabilities occupy partially orthogonal regions of parameter space. Mathematical reasoning capability is encoded largely in the attention mechanism through patterns that track numerical relationships, equation structure, and logical dependencies across multi-step derivations. Scientific domain knowledge is encoded primarily in the feed-forward layers through vocabulary, factual associations, and domain-specific inference patterns. These two functional locations in the network have relatively low overlap, which means merging adds both sets of capabilities with limited interference.

The phenomenon has been studied empirically in the context of instruction-following models merged with code generation models. Models instruction-fine-tuned to follow conversational prompts lack strong code generation ability, while models fine-tuned on code repositories lack natural conversational behavior. Merged models have been shown to combine conversational instruction-following with code generation capability, enabling the kind of interactive coding assistant behavior that neither source model exhibits alone.

However, capability composition is not always successful, and understanding the failure modes is as important as understanding the success cases.

When two capabilities require conflicting representations in shared parameters, merging degrades both. A clear example is two classification fine-tunes that use the same output head parameters but map very different classes. The output projection layer in both fine-tunes has been trained to map similar internal representations to different class probability distributions. Merging the output layers produces a confused mapping that serves neither task well, even if the rest of the network merges cleanly.

The key predictor of successful composition is whether the capabilities are complementary (they use different parameters and/or encode compatible information) versus competing (they both modify the same parameters in different directions). Before committing to a capability composition merge, it is worth computing the per-layer cosine similarity between the task vectors and checking whether the layers with high parameter overlap also show high sign conflict. If the top 20% of parameters by magnitude in both task vectors substantially overlap and show opposing signs, the merge is likely to degrade both capabilities in those layers.

A subtler failure mode is emergent capability conflict: two capabilities that each work well independently but interfere at inference time. A model that has been merged to be both highly creative (via a creative writing fine-tune) and highly factual (via a factual QA fine-tune) may produce outputs that are stylistically creative but factually unreliable, or factually accurate but stylistically flat, depending on the prompt. The capabilities are not compatible in the sense that activating one mode of generation tends to suppress the other, even though the weights for both are nominally present.

Frankenmerging (also called model stacking or layer grafting) is a variant of capability composition that avoids adding task vectors and instead physically assembles a new model by copying entire layers from different source models. A frankenmerge might take the first 16 layers from Model A (strong at language understanding) and the last 16 layers from Model B (strong at generation), producing a single 32-layer model that was never trained as a unit.

Frankenmerges are architecturally bold and empirically surprising: they sometimes work quite well despite having never seen end-to-end training. The mechanism is not fully understood, but the working hypothesis is that the early layers (embedding and low-level feature extraction) and late layers (output generation) are more modular than the middle layers, making cross-model assembly at these boundaries less disruptive.

There is an important distinction between frankenmerging and standard model stitching. In standard model stitching, a learned linear adapter is inserted at each layer boundary to align the representation spaces of the two models before continuing. Frankenmerging skips the adapter and simply concatenates the layers, betting that the representations are compatible enough at the cut point that no alignment is needed. This bet works surprisingly often, especially when both source models were fine-tuned from the same pre-trained base, because the pre-training creates a shared representational scaffold that persists through fine-tuning even as the weights diverge.

The shared pre-training scaffold is the key insight. When LLaMA 3 70B is fine-tuned by two different teams on two different tasks, the resulting models produce different outputs from the same input, but the intermediate representations at any given layer are more similar than they would be for models trained from different initializations. Both fine-tunes deformed the same pre-trained geometry, and they deformed it in task-specific directions while largely preserving the overall structure. This shared geometry is what allows layer grafting to work across fine-tunes of the same base.

A particularly creative use of frankenmerging is producing models of novel depth. If you have two 24-layer models and want a 32-layer model, you can take the first 16 layers from one model and the last 16 layers from the other. The resulting model has 32 layers despite neither source having that depth. While the resulting model will have different computational properties than a model natively trained at 32 layers, it can sometimes outperform either source on tasks that benefit from deeper processing.

This approach has been formalized in the "depth scaling via model merging" line of research, which showed that frankenmerged models can serve as strong initialization points for continued fine-tuning. Rather than training a 32-layer model from scratch (expensive), you assemble a 32-layer frankenmerge and fine-tune it for a fraction of the original training cost. The fine-tuning corrects for the discontinuities introduced at the grafting boundary, producing a coherent deep model at significantly reduced compute cost.

Manual capability composition requires intuition and luck. Automated methods treat the layer selection and coefficient assignment as an optimization problem. Two approaches have emerged.

Greedy layer selection evaluates each candidate configuration on a small benchmark and keeps the best-performing options, iterating until performance plateaus. The algorithm proceeds layer by layer: for each layer position, it tries all available source models and keeps the one that produces the best validation score when that layer is added to the current partial assembly. This is computationally cheap compared to exhaustive search but can get stuck in local optima because the optimal layer at position $l$ may depend on choices made at positions $l+1, l+2, \ldots$ that have not yet been fixed.

Evolutionary search (used by tools like EvoMerge) maintains a population of candidate merged models and applies mutation and crossover operators to explore the configuration space. Mutations change individual layer assignments or scaling coefficients; crossovers combine the configurations of two high-performing candidates into a new candidate that inherits components from both parents. Evolutionary search finds better solutions than greedy approaches but requires evaluating hundreds of merged models, which is feasible when evaluation is fast (e.g., using perplexity on a small held-out set) but expensive when evaluation requires running a full benchmark suite.

The EvoMerge paper showed that evolutionary search in the merging configuration space can discover non-intuitive configurations that manual design would miss. In one example, the best-performing configuration for a Japanese instruction-following task used layers from three different source models in a pattern that did not correspond to any simple "take the first half from A and second half from B" heuristic. The search discovered that certain middle layers from a third model provided representations that enabled better bridging between the understanding-oriented early layers and the generation-oriented late layers.

The practical constraint on evolutionary search is evaluation speed. Each candidate configuration requires at least a partial forward pass through the assembled model to score it. If the benchmark used for scoring requires thousands of examples, evaluating a population of 50 candidates for 20 generations means 1,000 evaluation runs. Using perplexity on a small held-out set (50-100 examples) as a proxy metric makes this feasible but introduces the risk that the proxy does not correlate perfectly with the target benchmark.

Evaluation is the bottleneck in practical merging work. Unlike fine-tuning, where loss curves provide continuous feedback, merging gives you a single candidate model per configuration. Poor evaluation leads to poor configuration choices, wasting the effort of the merge itself. Getting evaluation right is not a secondary concern, it is the core engineering challenge.

The first requirement for reliable merging evaluation is broad benchmark coverage. A single benchmark number is not sufficient because merged models often show asymmetric performance changes: they may improve on tasks related to one source model's fine-tune while regressing on tasks related to another. This asymmetry is invisible if you only measure one task.

A practical evaluation suite for a merged model should include:

• At least one benchmark for each source model's intended capability, measuring whether the merge preserved each fine-tune's core contribution
• A general language understanding benchmark (such as MMLU or HellaSwag) to detect global degradation in base capabilities, which can happen when conflicting task vectors suppress general knowledge
• A generation quality metric (perplexity, BERTScore, or MT-bench) to catch cases where the model generates incoherent text despite acceptable task scores
• An instruction-following evaluation if the models were instruction-tuned, because instruction-following is sensitive to the fine-tuning mix and can regress while task-specific metrics remain stable

The composition of this suite will vary by domain, but the principle holds: you need to measure all capabilities you care about, not only the capability you are trying to add.

Raw benchmark numbers are hard to compare across different evaluation suites and scales. A score of 78% on one benchmark and 63% on another does not immediately tell you whether the merge succeeded or failed relative to what was achievable. A useful normalization is:

where:

• $\text{score}_{\text{merged}}(t)$: the merged model's score on task $t$
• $\text{score}_{\text{base}}(t)$: the base model's score on task $t$ (the floor)
• $\text{score}_{\text{ft}}(t)$: the task-specific fine-tuned model's score on task $t$ (the ceiling)

This normalized score equals 1.0 if the merge perfectly preserves the fine-tune's performance, 0.0 if it fully reverts to the base, and can exceed 1.0 in rare cases where merging produces emergent improvement above the fine-tune's original level. Averaging normalized scores across all tasks gives an overall efficiency measure for the merge configuration.

The formula has an important interpretation: it measures what fraction of the fine-tuning benefit you retained. If the fine-tune improved task performance from 45% (base) to 75% (fine-tune), a normalized score of 0.8 means the merged model achieves $45 + 0.8 \times (75 - 45) = 69\%$. You retained 80% of the improvement the fine-tuning achieved, which is typically acceptable.

In practice, scores above 0.8 are considered good for multi-task merges, scores between 0.5 and 0.8 indicate moderate degradation that may be acceptable depending on deployment requirements, and scores below 0.5 suggest the merge is not working well and should be revisited with different algorithms or coefficients.

For a merge of two models, there is typically a tradeoff between performance on Task 1 and performance on Task 2. As $\lambda$ increases (weighting Task 1's fine-tune more heavily), Task 1 performance rises and Task 2 performance falls.

Plotting this tradeoff across all tested $\lambda$ values traces a curve that approximates the Pareto frontier of achievable configurations. The Pareto-optimal configurations are those where you cannot improve performance on one task without degrading the other. The goal of merging evaluation is to locate the Pareto frontier and then choose the configuration that best matches your deployment priorities.

In practice, the Pareto frontier for well-chosen model pairs lies above the diagonal (above the line connecting "all Task 1" to "all Task 2"), meaning merged models can be better at both tasks simultaneously than a simple interpolation would predict. This super-additive region is what makes merging useful rather than just a weighted average.

The diagonal baseline represents a model that selects outputs from either the Task 1 fine-tune or the Task 2 fine-tune at random, weighted by $\lambda$. Any merge configuration that achieves better performance on both tasks than this diagonal baseline produces a synergistic benefit from combining the two models' weights. The degree to which the Pareto frontier lies above this diagonal quantifies how much benefit the merge is providing beyond simple model selection.

When the Pareto frontier lies below the diagonal, the merge is actively harmful: you would do better by routing queries to one of the two source models based on task detection rather than using the merged model at all. This failure pattern typically occurs when the two tasks have high interference and the chosen merging algorithm is not handling it effectively.

Many benchmarks are slow to evaluate. Running MMLU, for example, requires thousands of forward passes with multi-choice completion. When searching over many merge configurations, you need faster proxies.

Two useful proxies have emerged from empirical experience with merging workflows.

Perplexity on task-specific data is fast and correlates reasonably well with downstream task performance. A merged model with lower perplexity on both tasks' held-out corpora is usually better than a merged model with higher perplexity on one, even before running the full benchmark. Computing perplexity requires a single forward pass per example with teacher-forced targets, which is faster than generating completions. On a single GPU, you can evaluate perplexity on 1,000 examples from each task in under a minute for a 7B-parameter model.

Calibration quality (the alignment between model confidence and actual accuracy) degrades in poorly merged models before task accuracy does. A merged model that becomes overconfident or underconfident is a signal of interference, even if task accuracy has not yet dropped measurably. Calibration can be measured with Expected Calibration Error (ECE) on a small labeled validation set. High ECE in a candidate configuration is a reliable early warning sign that the merge is causing internal representational conflicts.

A practical three-stage evaluation workflow combines these proxies efficiently. In the first stage, compute perplexity on held-out samples from each task's corpus for all candidate configurations. This takes seconds per configuration and immediately eliminates the worst options. In the second stage, run abbreviated benchmark evaluations (using 10-20% of benchmark examples) on the remaining top-ranked configurations. This narrows the candidate set further at moderate cost. In the third stage, run full benchmark evaluations only on the top three to five configurations from stage two. This staged approach reduces total evaluation compute by an order of magnitude compared to running full benchmarks on every configuration.

The staged approach also has the practical benefit of providing early stopping: if the first-stage perplexity numbers show that all configurations degrade significantly compared to the source models, you can abort the search and reconsider whether the chosen task pair is a good merging candidate, rather than running expensive benchmarks only to discover the same conclusion.

Beyond task-specific performance metrics, merged models can exhibit failure modes that do not show up in standard benchmarks but become apparent in production. Three categories of emergent failure are worth monitoring for.

Consistency failures occur when a merged model gives contradictory answers to the same question phrased differently. A model that correctly answers "Paris is the capital of France" as a multiple-choice question but hallucinates "Lyon" when asked in open-ended generation may have merged models with conflicting factual associations for that piece of knowledge. Consistency can be tested by asking the same factual queries in multiple formats and checking agreement.

Style inconsistency is a failure mode specific to style merges, where the model switches between two styles within a single long-form output. The early tokens in a completion may exhibit one style while later tokens shift to the other, because the model's internal state has evolved in a way that activates a different style register. This can be detected by splitting long outputs into chunks and running the style classifier on each chunk separately, checking for within-output variance.

Instruction conflict occurs when merged instruction-tuned models have conflicting expectations about how to interpret certain prompt patterns. One source model may have been trained to refuse certain request types with a specific decline phrasing, while another may have been trained to fulfill similar requests in a different context. The merged model may exhibit unpredictable behavior on borderline requests that fall in the overlap region of the two training distributions.

This section provides a practical implementation of multi-task merging and evaluation using PyTorch. We use small simulated parameter vectors suitable for a local machine, but the same patterns apply at any scale with real model checkpoints.

We start by defining simulated model weights representing a base model and two independently fine-tuned models. The simulation uses sparse task vectors (each fine-tune modifies only a subset of parameters) to mimic the behavior of real fine-tunes, where gradient updates are concentrated on the parameters most relevant to each task.

The cosine similarity between the two task vectors indicates how much interference to expect. A near-zero value suggests the two fine-tunes modified largely orthogonal parameters, which is favorable for merging. The overlap count tells you how many parameters both fine-tunes modified simultaneously, which is the region where sign conflicts can occur.

We implement the basic task arithmetic merge across a range of scaling coefficients to trace the performance tradeoff. The evaluate_performance function acts as a proxy for real benchmark evaluation by measuring how close the merged model is to the target fine-tune in parameter space. In a real workflow, you would replace this with actual benchmark scoring.

The best combined lambda balances the two tasks. At lambda = 1.0, the merge uses the full task vectors, which may cause interference depending on overlap between the fine-tunes. Notice that the optimal lambda is often not 1.0: scaling the task vectors down slightly reduces interference while retaining most of the capability contribution from each fine-tune.

Next, we implement a simplified version of TIES merging, which resolves sign conflicts before summing task vectors. The three-step procedure (trim, elect, disjoint merge) handles the case where multiple task vectors modify the same parameter in opposite directions.

TIES merging reduces interference by eliminating conflicting signals before aggregation. The improvement is most pronounced when the two task vectors have substantial parameter overlap with opposing signs. When the task vectors are nearly orthogonal (low overlap), TIES and task arithmetic tend to produce similar results because there are few conflicts to resolve.

We now compute the normalized performance score introduced earlier, which expresses the merged model's performance as a fraction of the gap between the base and fine-tuned models. This normalization makes it easy to compare results across different evaluation setups.

A normalized score of 1.0 means the merged model is as good as training a separate fine-tune for that task. Scores above 0.8 are generally considered acceptable for production use cases. The average normalized score across tasks provides a single figure of merit for comparing merge configurations, and it can be extended straightforwardly to three or more tasks by simply averaging over all task scores.

The key parameters for multi-task merging are:

• lambda ($\lambda$): The scaling coefficient for task vectors. Higher values apply more of each fine-tune's changes. Typical range: 0.3 to 0.8 for two-task merges.
• trim_fraction (TIES): The fraction of low-magnitude parameters to zero out before merging. Higher values reduce interference but may discard useful small adaptations. Typical range: 0.1 to 0.3.
• density (DARE): The fraction of task vector parameters to retain after random sparsification. Lower values reduce interference but require higher lambda to compensate. Typical range: 0.5 to 0.9.

Model merging is a powerful technique with real limitations that deserve serious attention before you commit to a merging strategy for production use.

The most fundamental limitation is that merging cannot recover capabilities that were lost during fine-tuning. When a model is fine-tuned aggressively on a narrow task, the base model's general capabilities can degrade significantly, a phenomenon called catastrophic forgetting that was discussed in the fine-tuning fundamentals chapters. Merging the fine-tuned model back with the base model can restore some of those capabilities, because the base model's weights still contain the general representations. But this only works if the fine-tuned model still contains the relevant intermediate representations, which is not guaranteed when fine-tuning has been severe. A heavily fine-tuned model may have restructured its internal representations so thoroughly that simple interpolation with the base cannot reconstruct the original general-purpose behavior.

The compatibility requirement is strict. All models being merged must share the same architecture and tokenizer. You cannot merge models of different sizes (a 7B model and a 13B model), different attention head configurations (different numbers of heads per layer), or models trained with different tokenizers. The tokenizer constraint is particularly important: even if two models share the same architecture but were trained with different vocabularies, their embedding matrices are not aligned, and merging embedding layers across incompatible vocabularies produces nonsense. This limits merging to the growing but still bounded ecosystem of publicly available fine-tunes of specific popular base models. LLaMA 3, Mistral, and Gemma have the richest merging ecosystems, with hundreds of community fine-tunes available on Hugging Face that can theoretically be combined.

The strict architecture compatibility requirement also means that model merging cannot be used to combine proprietary closed-source models with open fine-tunes. You need access to the raw weight tensors, which closed API-only models do not provide. This positions merging as fundamentally an open-source model technique, at least for now.

Evaluation is hard in a way that training is not. During fine-tuning, loss curves provide continuous feedback about whether the model is improving. During a merging sweep, you get one data point per configuration, and that data point may not reveal all relevant failure modes. Merged models can perform well on standard benchmarks while failing in unexpected ways on production traffic, particularly when the fine-tunes being merged represent somewhat different domains than the evaluation benchmarks. A merged model that scores well on MMLU and HellaSwag may still generate inconsistent outputs on edge-case prompts that activate conflicting representations from the two source models.

Another limitation that is often overlooked is the sensitivity of merging to the fine-tuning hyperparameters of the source models. Two fine-tunes of the same base on the same data with different learning rates will produce task vectors of very different magnitudes. When you merge these, the higher-magnitude task vector will dominate even if you set equal $\lambda$ values, because it contributes larger absolute updates to each parameter. Normalizing task vectors by their L2 norm before merging can mitigate this, but it introduces a new hyperparameter choice and can cause its own artifacts.

Finally, merging is not a substitute for domain-specific fine-tuning when the target task requires knowledge or reasoning patterns absent from all available fine-tunes. Merging recombines what exists in the source models. It does not generate new capabilities from nothing. If no available fine-tune contains examples of reasoning about a specific technical domain, merging fine-tunes from adjacent domains may provide a useful starting point, but it will not achieve the performance that direct fine-tuning on domain-specific data would provide.

Despite these constraints, the practical value of merging is significant. It offers a route to multi-capability models that would cost substantial GPU-hours to train, using only inference-time compute for the merge operation itself. A multi-task merge that combines three fine-tunes into a single model using task arithmetic takes seconds on a standard GPU, while training an equivalent multitask model from scratch might require days. As the ecosystem of fine-tuned models continues to grow, the space of useful merges will expand correspondingly, and the techniques for diagnosing and resolving interference will improve.

Model merging methods continue to change. Recent work has explored learned merging coefficients (where a small optimization loop automatically finds the best $\lambda$ values using a few gradient steps on validation data), structured sparsification approaches that go beyond TIES and DARE, and methods that merge only specific components (attention heads versus feed-forward blocks) rather than all parameters uniformly. These extensions have not yet reached the maturity of the core task arithmetic and TIES approaches, but they represent directions where the field is heading.

Looking ahead, the techniques covered in this chapter and the preceding model compression chapters represent the current state of post-training model manipulation. The next section of the book turns to reasoning: how models can be prompted or trained to exhibit structured, multi-step reasoning, and what the limits of that reasoning are.

Model merging applications span four main domains: multi-task merging, style merging, capability composition, and evaluation.

Multi-task merging combines independently fine-tuned models into a single generalist without requiring access to task data or re-training. The task arithmetic formula $\theta_{\text{merged}} = \theta_{\text{base}} + \sum_t \lambda_t \tau_t$ is the core operation, with TIES and DARE variants reducing interference from conflicting task vectors. Before merging, computing pairwise cosine similarities between task vectors provides an early signal about which merges are likely to succeed. Optimal $\lambda$ values typically fall between 0.3 and 0.8, and a task loss below 5% (meaning the merged model performs at 95% of the dedicated fine-tune's level) is generally considered acceptable.

Style merging applies merging to blend behavioral and stylistic characteristics rather than task competencies. Layer-wise merging assigns different blend coefficients to different transformer depths, weighting later layers more heavily because high-level stylistic information concentrates there. SLERP offers a geometrically principled interpolation that preserves vector norms, avoiding the scale reduction that standard linear interpolation introduces at intermediate blend values and that can produce flat-feeling outputs.

Capability composition assembles models with novel abilities by combining specializations that occupy partially orthogonal parameter space. Success depends on whether the target capabilities are complementary (using different parameters) or competing (modifying the same parameters in conflicting directions). Frankenmerging extends capability composition to physical layer assembly, exploiting the shared representational scaffold that pre-training creates across all models initialized from the same base. Evolutionary search automates the configuration search by maintaining a population of candidate configurations and applying mutation and crossover operators.

Merging evaluation requires broad benchmark coverage, normalized performance scores that express merged model quality relative to both base and fine-tuned ceilings, and Pareto analysis to understand the tradeoff surface between tasks. A normalized score of 1.0 means the merge perfectly preserved the fine-tune's benefit, while 0.0 means full regression to base performance. Fast proxies including perplexity on task-specific held-out data and calibration quality enable efficient configuration search before running expensive full evaluations, enabling a staged workflow that reduces total evaluation compute by an order of magnitude.

Ready to test your understanding? Take this quick quiz to reinforce what you've learned about model merging applications.