Model Merging Applications › Capability Composition

Capability composition goes beyond combining task-specific fine-tunes to assembling models with capabilities not present in any single source model by combining specialized base capabilities. This is perhaps the most technically ambitious application of merging, and also the one that has attracted the most research attention in recent years.

One prominent example is combining a model fine-tuned for mathematical reasoning with a model fine-tuned on scientific domain knowledge. Neither model alone can reason about domain-specific quantitative problems: the math model lacks the domain vocabulary and conceptual framework, while the science model lacks the systematic quantitative reasoning patterns. A merge of both can potentially exhibit emergent ability to apply mathematical reasoning within the scientific domain, producing a model that can work through numerical problems in biology, chemistry, or physics with both technical accuracy and domain-appropriate framing.

This works because the two capabilities occupy partially orthogonal regions of parameter space. Mathematical reasoning capability is encoded largely in the attention mechanism through patterns that track numerical relationships, equation structure, and logical dependencies across multi-step derivations. Scientific domain knowledge is encoded primarily in the feed-forward layers through vocabulary, factual associations, and domain-specific inference patterns. These two functional locations in the network have relatively low overlap, which means merging adds both sets of capabilities with limited interference.

The phenomenon has been studied empirically in the context of instruction-following models merged with code generation models. Models instruction-fine-tuned to follow conversational prompts lack strong code generation ability, while models fine-tuned on code repositories lack natural conversational behavior. Merged models have been shown to combine conversational instruction-following with code generation capability, enabling the kind of interactive coding assistant behavior that neither source model exhibits alone.

However, capability composition is not always successful, and understanding the failure modes is as important as understanding the success cases.

When two capabilities require conflicting representations in shared parameters, merging degrades both. A clear example is two classification fine-tunes that use the same output head parameters but map very different classes. The output projection layer in both fine-tunes has been trained to map similar internal representations to different class probability distributions. Merging the output layers produces a confused mapping that serves neither task well, even if the rest of the network merges cleanly.

The key predictor of successful composition is whether the capabilities are complementary (they use different parameters and/or encode compatible information) versus competing (they both modify the same parameters in different directions). Before committing to a capability composition merge, it is worth computing the per-layer cosine similarity between the task vectors and checking whether the layers with high parameter overlap also show high sign conflict. If the top 20% of parameters by magnitude in both task vectors substantially overlap and show opposing signs, the merge is likely to degrade both capabilities in those layers.

A subtler failure mode is emergent capability conflict: two capabilities that each work well independently but interfere at inference time. A model that has been merged to be both highly creative (via a creative writing fine-tune) and highly factual (via a factual QA fine-tune) may produce outputs that are stylistically creative but factually unreliable, or factually accurate but stylistically flat, depending on the prompt. The capabilities are not compatible in the sense that activating one mode of generation tends to suppress the other, even though the weights for both are nominally present.

Frankenmerging (also called model stacking or layer grafting) is a variant of capability composition that avoids adding task vectors and instead physically assembles a new model by copying entire layers from different source models. A frankenmerge might take the first 16 layers from Model A (strong at language understanding) and the last 16 layers from Model B (strong at generation), producing a single 32-layer model that was never trained as a unit.

Frankenmerges are architecturally bold and empirically surprising: they sometimes work quite well despite having never seen end-to-end training. The mechanism is not fully understood, but the working hypothesis is that the early layers (embedding and low-level feature extraction) and late layers (output generation) are more modular than the middle layers, making cross-model assembly at these boundaries less disruptive.

There is an important distinction between frankenmerging and standard model stitching. In standard model stitching, a learned linear adapter is inserted at each layer boundary to align the representation spaces of the two models before continuing. Frankenmerging skips the adapter and simply concatenates the layers, betting that the representations are compatible enough at the cut point that no alignment is needed. This bet works surprisingly often, especially when both source models were fine-tuned from the same pre-trained base, because the pre-training creates a shared representational scaffold that persists through fine-tuning even as the weights diverge.

The shared pre-training scaffold is the key insight. When LLaMA 3 70B is fine-tuned by two different teams on two different tasks, the resulting models produce different outputs from the same input, but the intermediate representations at any given layer are more similar than they would be for models trained from different initializations. Both fine-tunes deformed the same pre-trained geometry, and they deformed it in task-specific directions while largely preserving the overall structure. This shared geometry is what allows layer grafting to work across fine-tunes of the same base.

A particularly creative use of frankenmerging is producing models of novel depth. If you have two 24-layer models and want a 32-layer model, you can take the first 16 layers from one model and the last 16 layers from the other. The resulting model has 32 layers despite neither source having that depth. While the resulting model will have different computational properties than a model natively trained at 32 layers, it can sometimes outperform either source on tasks that benefit from deeper processing.

This approach has been formalized in the "depth scaling via model merging" line of research, which showed that frankenmerged models can serve as strong initialization points for continued fine-tuning. Rather than training a 32-layer model from scratch (expensive), you assemble a 32-layer frankenmerge and fine-tune it for a fraction of the original training cost. The fine-tuning corrects for the discontinuities introduced at the grafting boundary, producing a coherent deep model at significantly reduced compute cost.

Manual capability composition requires intuition and luck. Automated methods treat the layer selection and coefficient assignment as an optimization problem. Two approaches have emerged.

Greedy layer selection evaluates each candidate configuration on a small benchmark and keeps the best-performing options, iterating until performance plateaus. The algorithm proceeds layer by layer: for each layer position, it tries all available source models and keeps the one that produces the best validation score when that layer is added to the current partial assembly. This is computationally cheap compared to exhaustive search but can get stuck in local optima because the optimal layer at position $l$ may depend on choices made at positions $l+1, l+2, \ldots$ that have not yet been fixed.

Evolutionary search (used by tools like EvoMerge) maintains a population of candidate merged models and applies mutation and crossover operators to explore the configuration space. Mutations change individual layer assignments or scaling coefficients; crossovers combine the configurations of two high-performing candidates into a new candidate that inherits components from both parents. Evolutionary search finds better solutions than greedy approaches but requires evaluating hundreds of merged models, which is feasible when evaluation is fast (e.g., using perplexity on a small held-out set) but expensive when evaluation requires running a full benchmark suite.

The EvoMerge paper showed that evolutionary search in the merging configuration space can discover non-intuitive configurations that manual design would miss. In one example, the best-performing configuration for a Japanese instruction-following task used layers from three different source models in a pattern that did not correspond to any simple "take the first half from A and second half from B" heuristic. The search discovered that certain middle layers from a third model provided representations that enabled better bridging between the understanding-oriented early layers and the generation-oriented late layers.

The practical constraint on evolutionary search is evaluation speed. Each candidate configuration requires at least a partial forward pass through the assembled model to score it. If the benchmark used for scoring requires thousands of examples, evaluating a population of 50 candidates for 20 generations means 1,000 evaluation runs. Using perplexity on a small held-out set (50-100 examples) as a proxy metric makes this feasible but introduces the risk that the proxy does not correlate perfectly with the target benchmark.