Model Merging Task Arithmetic Combining Fine-Tuned Models Without Retraining
# Model Merging & Task Arithmetic: Combining Fine-Tuned Models Without Retraining
## Introduction & Motivation
Fine-tuning a large pretrained model on a new task or dataset traditionally produces a single specialized model, distinct from the base model and from every other fine-tuned variant derived from it. As the ecosystem of publicly available fine-tuned checkpoints has exploded, one for each downstream task, language, instruction-following style, or preference alignment objective, a natural question has emerged: rather than training a new model from scratch to combine several such specializations, can the weights of independently fine-tuned models simply be combined directly, without any further gradient-based training, to produce a single model that exhibits multiple specialized capabilities at once?
Model merging answers this question affirmatively across a surprisingly broad range of settings. The foundational empirical observation, formalized as task arithmetic, is that the weight difference between a fine-tuned model and its shared base model, termed a task vector, behaves in many practical respects like a genuine vector: task vectors for different, sufficiently independent tasks can often be added together and applied to the base model to produce a single model competent at all of the constituent tasks, and a task vector can be subtracted to suppress or unlearn the specific behavior it encodes, all without touching the training data or running any additional optimization. This is a remarkable property given that neural network loss landscapes are highly non-convex and there is no a priori mathematical guarantee that linear operations in weight space should correspond to any coherent operation in function space.
Model merging has since grown from this simple additive task-arithmetic observation into a family of increasingly sophisticated techniques, including spherical interpolation that better respects the geometry of high-dimensional weight space, and sign-conflict-resolving methods such as TIES-merging and DARE that address the practical reality that naively averaging many task vectors causes destructive interference between updates that disagree in sign or overlap in the parameters they modify. These techniques matter practically because they offer a training-free, dataset-free, and often remarkably cheap alternative to multi-task fine-tuning or model distillation for combining capabilities, and because understanding when and why merging works, and when it breaks down, sheds useful light on the geometry of the fine-tuning loss landscape itself.
## Core Concepts & Theory
A task vector is defined as the element-wise difference between a fine-tuned model's weights and the shared base (pretrained) model's weights from which it was fine-tuned, $ au = heta_{ ext{finetuned}} - heta_{ ext{base}}$. This vector is understood as encoding, in weight space, whatever behavioral change the fine-tuning process induced relative to the base model. The central empirical claim of task arithmetic is that these vectors compose approximately linearly under two operations directly analogous to vector arithmetic: addition of two or more task vectors to the base model tends to produce a model exhibiting a combination of the corresponding behaviors, and negation of a task vector, subtracting it from a model that already exhibits the behavior, tends to suppress that specific behavior while leaving largely unrelated capabilities intact.
This approximate linearity is not universal; it holds most reliably when the task vectors involved are "low-interference," a property that has been linked empirically and partially theoretically to the parameter updates for different tasks occupying largely distinct, low-overlap subsets or directions within the high-dimensional weight space, so that adding one task's update does not substantially disturb the weight components another task's update relies on. Fine-tuning runs that stay within a comparatively small region of weight space around a shared, capable pretrained initialization, as is typical of modern parameter-efficient and full fine-tuning regimes applied to large pretrained foundation models, tend to produce task vectors with this low-interference property far more often than models trained from independent random initializations, which is a large part of why model merging became practically viable primarily in the era of large pretrained foundation models rather than earlier in deep learning's history.
Naive linear averaging, the simplest possible merging operation, computes a weighted mean of two or more full model weight vectors (or, equivalently, applies the mean of their task vectors to the shared base). While simple and often surprisingly effective for merging two models fine-tuned similarly on related tasks, it has well-documented failure modes when merging more than a couple of models or models whose task vectors substantially overlap in the parameters they modify but disagree in the sign of their update, a phenomenon termed sign interference, since directly averaging opposing-sign updates causes them to partially or fully cancel, discarding information that any individual task vector correctly encoded.
## Mathematical Formulation
Given a shared base model with weights $ heta_{ ext{base}}$ and $n$ independently fine-tuned models with weights $ heta_1, \ldots, heta_n$, each task vector is
$$ au_i = heta_i - heta_{ ext{base}}, \quad i = 1, \ldots, n $$
Task arithmetic combines these by simple addition, optionally with per-task scaling coefficients $\lambda_i$ to control each task's relative influence on the merged model,
$$ heta_{ ext{merged}} = heta_{ ext{base}} + \sum_{i=1}^{n} \lambda_i au_i $$
with $\lambda_i = 1$ for straightforward addition and negative $\lambda_i$ for behavior suppression (negation). Naive linear model averaging is the special case where task vectors are not computed at all and the merged weights are directly a convex combination, $ heta_{ ext{merged}} = \sum_i w_i heta_i$ with $\sum_i w_i = 1$, which is mathematically equivalent to task arithmetic with $\lambda_i = w_i$ whenever all models share the same base.
Spherical linear interpolation (SLERP) between two weight vectors $ heta_1, heta_2$ addresses a specific geometric shortcoming of linear averaging: because high-dimensional weight vectors of similar norm but differing direction have a linear midpoint whose norm shrinks according to the cosine of the angle between them, naive averaging systematically reduces the effective magnitude of the merged weights whenever the two models point in substantially different directions. SLERP instead interpolates along the great-circle arc connecting the two (normalized) directions,
$$ ext{slerp}( heta_1, heta_2, t) = \frac{\sin((1-t)\Omega)}{\sin \Omega} heta_1 + \frac{\sin(t\Omega)}{\sin \Omega} heta_2, \quad \Omega = \arccos\left( \frac{ heta_1 \cdot heta_2}{\| heta_1\| \| heta_2\|} ight) $$
which, when combined with a linear interpolation of the two vectors' norms, preserves a consistent effective weight magnitude regardless of the angle between the two models, avoiding the destructive-cancellation shrinkage that plain linear averaging exhibits.
TIES-merging (Trim, Elect Sign, and Merge) is designed specifically to combine more than two task vectors while resolving sign interference. It proceeds in three steps: first, trimming, in which each task vector is sparsified by retaining only its top-magnitude entries (by absolute value) and zeroing the rest, removing noisy, low-magnitude updates that contribute little signal but add interference; second, electing a sign, in which, for each individual parameter, the algorithm determines which sign (positive or negative) has greater total support across the trimmed task vectors, either by count of agreeing entries or by their summed magnitude,
$$ \gamma_j = ext{sign}\left( \sum_{i=1}^{n} \hat au_{i,j} ight) $$
where $\hat au_i$ denotes the trimmed task vector; and third, a disjoint merge, in which the final value for each parameter is computed by averaging only the entries from task vectors that agree with the elected sign at that parameter, discarding the disagreeing entries entirely rather than allowing them to partially cancel the agreeing ones,
$$ au_{ ext{merged}, j} = \frac{1}{|\{i : ext{sign}(\hat au_{i,j}) = \gamma_j\}|} \sum_{i : ext{sign}(\hat au_{i,j}) = \gamma_j} \hat au_{i,j} $$
DARE (Drop And REscale) offers a complementary insight applicable as a preprocessing step before any merge method: it randomly drops a large fraction (often 90 percent or more) of each task vector's entries and rescales the remaining entries by dividing by the retention probability to preserve the task vector's expected magnitude, exploiting the empirical observation that fine-tuning updates are highly redundant and that most of a task vector's individual parameter changes are not, in isolation, necessary to preserve most of the fine-tuned behavior.
## Advanced Theory & Extensions
Linear mode connectivity, the empirical observation that two models fine-tuned from the same pretrained initialization are often connected by a path of comparatively low loss when linearly interpolated in weight space, is widely cited as the theoretical grounding for why model merging works at all in the fine-tuning regime, in contrast to models trained from independent random initializations, which are generally not linearly mode connected and whose weights cannot be usefully averaged without an intermediate re-alignment (permutation matching) step, since different random initializations can converge to functionally similar but differently permuted internal representations. Fine-tuning starting from a shared, already highly structured pretrained initialization appears to keep different task-specific solutions within the same broad basin of the loss landscape, which is the geometric condition under which linear (and spherical) interpolation between them tends to remain low-loss.
Multi-task model merging can be understood as an approximation to true joint multi-task fine-tuning, and a body of work has examined when this approximation is tight, generally finding that merged models approach, but rarely exceed, the performance of a model explicitly fine-tuned jointly on all the constituent tasks' data, particularly as the number of merged tasks grows and the likelihood of destructive parameter interference increases; this has motivated evaluating merging quality not just against the unmerged base model but explicitly against the (more expensive) joint fine-tuning alternative as an upper-bound reference point. Task vector negation for targeted unlearning or safety-behavior suppression has been explored as a lightweight alternative to more expensive alignment retraining, subtracting a task vector representing an undesirable behavior (learned, for instance, by deliberately fine-tuning a copy of the model to exhibit that behavior) from a production model, though this approach's reliability depends heavily on how cleanly the undesirable behavior's task vector can be isolated from unrelated capabilities the base model needs to retain.
## Computational Considerations
Model merging's central practical appeal is computational: producing a merged model costs essentially the price of a handful of element-wise tensor operations on already-existing checkpoints, orders of magnitude cheaper than any approach requiring gradient computation, data access, or GPU training time, and merging can typically be performed on CPU or with minimal GPU memory even for models with tens of billions of parameters, since only the weight tensors themselves, not activations, gradients, or optimizer state, need to be held in memory.
Memory requirements scale with the number of full model checkpoints that must be loaded simultaneously (or streamed sequentially) during the merge computation, which becomes a practical bottleneck when merging many large models at once; production merging pipelines commonly process the merge layer by layer or tensor by tensor, loading only the corresponding slice from each source checkpoint at a time rather than holding every full model in memory simultaneously. TIES-merging's trimming and sign-election steps add modest additional computation, roughly linear in the number of parameters and the number of models being merged, but remain negligible relative to any training-based alternative, and DARE's random dropping step is similarly cheap, requiring only a random mask generation and an elementwise rescale per task vector.
## Practical Implementation Strategies
Merging is most reliable, and most commonly applied in practice, to models that share an identical architecture and were fine-tuned from the same base checkpoint, since task vectors are only mathematically well-defined relative to a common reference point, and merging models with different base checkpoints, even nominally similar ones, typically produces substantially degraded results due to the lack of any linear mode connectivity guarantee across separately pretrained models. Before merging more than two or three task vectors, practitioners should assess pairwise sign-conflict rates and cosine similarity between task vectors to anticipate whether naive averaging is likely to suffice or whether a sign-resolving method such as TIES-merging is needed; a high fraction of parameters with disagreeing signs across task vectors is a strong practical signal that naive merging will underperform.
Per-task scaling coefficients in the task-arithmetic formulation are an important and often underused tuning lever: rather than applying every task vector at full strength ($\lambda_i = 1$), searching over a shared or per-task scaling factor on a small validation set for each constituent task frequently recovers substantial performance that naive full-strength addition leaves on the table, since interference effects are often reduced, though not eliminated, by moderately down-weighting each individual task vector's contribution to the sum. When the goal is targeted behavior suppression via negation, it is good practice to validate the resulting model on a broad capability benchmark in addition to the specific suppressed behavior, since negation can have unpredictable side effects on capabilities that were more entangled with the negated behavior's task vector than initially assumed.
Merging is not limited to combining full fine-tuned checkpoints; it applies equally well to low-rank adapters, where merging several LoRA adapters trained for different tasks or styles, either by direct weight-space combination of the adapters or by applying analogous sign-resolution techniques, offers an especially lightweight way to combine many small, cheaply-trained specializations without ever materializing or storing the full combined dense weight difference until deployment time.
## Benchmark Datasets & Evaluation
Model merging research commonly evaluates on suites spanning multiple, clearly delineated tasks simultaneously, such as combinations of GLUE or SuperGLUE natural language understanding tasks, image classification benchmarks spanning several distinct domains for vision model merging, and instruction-following and safety benchmarks for merging chat-oriented language model variants, specifically because the interesting merging question is not single-task performance but retained performance across every constituent task simultaneously relative to each task's individually fine-tuned specialist model. A standard reporting convention is the normalized average performance across all merged tasks, expressed as a fraction of the average performance achieved by the individually fine-tuned specialist models on their own respective tasks, giving a single number that captures how much capability, on average, survives the merging process relative to the unmerged ceiling.
Held-out capability retention benchmarks, general-purpose evaluation suites unrelated to any of the specifically merged tasks, are equally important to report alongside task-specific merge performance, since a merge that achieves strong performance on its target tasks but silently degrades broad general capability (a form of catastrophic forgetting induced by the merge operation itself) would not be revealed by task-specific metrics alone. For merging applied to safety or alignment-related task vectors specifically, red-teaming-style adversarial evaluation is used in addition to standard benchmarks, since the practical question for a negated-behavior merge is not average-case performance but worst-case robustness of the suppression.
## Key Challenges & Limitations
Merging quality degrades, often sharply, as the number of simultaneously merged tasks grows, since the probability that at least some parameters experience meaningful sign or magnitude interference across many independently fine-tuned task vectors increases with the number of tasks being combined; sign-resolving methods such as TIES-merging substantially mitigate but do not eliminate this degradation, and very large numbers of merged tasks (dozens or more) remain a genuinely difficult regime for any current training-free merging method. Task vectors are not guaranteed to be behaviorally isolable: a task vector derived from fine-tuning on a specific dataset frequently encodes some entangled mixture of the intended task-specific behavior and incidental changes to more general capabilities, meaning both addition (combining tasks) and negation (suppressing a behavior) can produce side effects on ostensibly unrelated capabilities that are difficult to predict in advance without empirical evaluation.
Merging fundamentally requires a shared base model and, in practice, closely related fine-tuning procedures (similar learning rates, similar number of training steps, similar data scale) to produce task vectors that remain within the same broadly linearly-connected region of weight space; merging checkpoints that diverged substantially from their shared base, for instance after very long or aggressive fine-tuning runs, tends to produce markedly worse results than merging lightly fine-tuned or parameter-efficiently fine-tuned checkpoints, which is part of why model merging has flourished particularly in ecosystems built around parameter-efficient fine-tuning methods that inherently keep task vectors small. Finally, merging offers no formal guarantee of the resulting model's behavior; unlike joint multi-task training, which directly optimizes a combined objective, a merged model's multi-task competence and safety properties are empirically observed rather than provably derived from the merge procedure, making thorough post-merge evaluation a necessity rather than an optional precaution.
## Hyperparameter Tuning
The per-task scaling coefficient (or a single shared scaling coefficient applied uniformly to a sum of task vectors) is the most consequential and most commonly tuned hyperparameter in task arithmetic, typically searched over a modest grid (for instance, values between 0.3 and 1.0) against a small validation set per task, since both too-small and too-large coefficients degrade merged performance: too small under-expresses each task's specialization, while too large amplifies interference effects between task vectors. TIES-merging's trim fraction, the proportion of each task vector's smallest-magnitude entries zeroed out before sign election, trades off noise reduction against information loss; overly aggressive trimming discards genuine, if individually small, task-relevant signal, while insufficient trimming allows noisy near-zero entries to inappropriately sway the sign-election vote at a given parameter, particularly when merging many task vectors.
DARE's drop rate similarly requires validation, since it exploits genuine redundancy in fine-tuning updates but has a task- and model-dependent limit beyond which dropping too large a fraction of a task vector's entries begins to measurably degrade the represented behavior even after magnitude rescaling; published results suggest surprisingly high viable drop rates (90 percent or more) for many large language model fine-tuning task vectors, though this should not be assumed to transfer uniformly across architectures or fine-tuning regimes without validation. SLERP's interpolation parameter $t$, when merging exactly two models, directly controls the relative balance between them and is typically set based on validation performance across the interpolation path rather than fixed a priori at the naive midpoint value of 0.5, since the optimal blend point is rarely exactly equidistant between two models with different individual capability profiles.
## Real-World Applications & Case Studies
Open-source language model communities have used task-arithmetic-style merging extensively to combine instruction-following, domain-specialized (such as coding or mathematics), and stylistic fine-tunes derived from a shared open base model, producing merged checkpoints that inherit multiple specializations without requiring access to the original constituent training datasets, which are frequently unavailable or prohibitively expensive to reproduce; this practice has become common enough that dedicated merged-model leaderboards and merge-recipe-sharing conventions have emerged within these communities. Vision model merging has been applied to combine classifiers or feature extractors independently fine-tuned on different domains or datasets (for instance, combining a model fine-tuned for satellite imagery with one fine-tuned for medical imaging, both from a shared pretrained vision backbone) into a single multi-domain model without needing simultaneous access to both original training sets, which is particularly valuable when the constituent datasets are governed by different data-sharing or privacy restrictions that prevent joint training.
Task vector negation has been explored as a lightweight technique for removing specific undesirable behaviors acquired during fine-tuning, such as excessive verbosity, specific factual errors reinforced by a particular fine-tuning dataset, or narrowly scoped toxic outputs, by fine-tuning a small auxiliary model specifically to exhibit the undesirable behavior more strongly and then subtracting its task vector from the production model, offering a targeted alternative to full retraining when the undesirable behavior can be cleanly isolated. Federated and collaborative model development scenarios, in which different organizations or teams each fine-tune a shared base model independently, sometimes on private, non-shareable data, have used merging as a mechanism to combine the resulting specializations into a single model without any party needing to share their underlying training data, an application that directly leverages merging's defining property of requiring only model weights, not training data, as input.
## Integration with Other Methods
Model merging interacts closely with parameter-efficient fine-tuning methods, since a LoRA adapter is itself already a compact representation of a task-specific weight delta relative to a shared frozen base, making adapters a particularly natural and storage-efficient unit for merging; multi-adapter serving systems, which already maintain many small task-specific adapters against a shared base for efficient simultaneous serving, can apply merging techniques either at request time (composing several adapters dynamically) or ahead of time (baking a fixed merge into a single deployed adapter) depending on whether the combination of tasks needed is known in advance. Model merging is also increasingly used as a lightweight post-processing step following reinforcement learning from human feedback or direct preference optimization pipelines, where separately trained reward-optimized and capability-preserving checkpoints are merged to recover a balance between alignment strength and general capability retention that a single end-to-end training run might not have achieved as cleanly.
Knowledge distillation and model merging are complementary rather than competing techniques: distillation transfers behavior from a teacher to a differently-sized or differently-architected student through training on generated outputs, while merging combines models that already share an architecture and lineage through direct weight-space operations, and the two are sometimes used together, for instance distilling several specialist teachers into intermediate same-architecture checkpoints specifically to make them mergeable, before applying a merge operation to combine them into a final deployed model.
## Future Research Directions
A more principled theoretical account of exactly when and why task vectors compose approximately linearly remains an active research question, since current understanding rests substantially on empirical observation of linear mode connectivity and low-interference conditions rather than a complete derivation from first principles of neural network training dynamics; a stronger theory would ideally predict, in advance of any merge attempt, which combinations of tasks and fine-tuning procedures are likely to merge well without requiring costly empirical trial and error across scaling coefficients and merge methods. Automated, learned merging, in which the combination weights, trimming thresholds, or even which parameters to merge at all are themselves optimized against a small amount of validation data rather than manually tuned or fixed by a hand-designed algorithm, is an active direction that trades away some of merging's zero-training-cost appeal in exchange for potentially substantially better merge quality, particularly as the number of tasks being combined grows.
Extending merging techniques to models that do not share an identical architecture, through techniques such as weight or activation alignment prior to merging, remains substantially less mature than same-architecture merging and is an important direction for broadening merging's applicability beyond ecosystems built around a single shared base model family. Finally, as merging is increasingly applied to safety-relevant task vectors, both for combining alignment techniques and for suppressing undesirable behaviors via negation, developing more rigorous evaluation standards and theoretical guarantees for merge-induced safety properties, rather than relying purely on empirical post-hoc red-teaming, is an increasingly pressing need given how cheaply and widely merging can be applied by parties without the resources to conduct exhaustive safety evaluation themselves.
## Summary & Key Takeaways
Model merging exploits the empirical observation that task vectors, the weight-space difference between a fine-tuned model and its shared pretrained base, compose approximately linearly under addition and negation, enabling multiple independently fine-tuned specializations to be combined into a single model, or an unwanted behavior to be suppressed, entirely without gradient-based retraining or access to any training data. This property rests on linear mode connectivity between models fine-tuned from a shared initialization and holds most reliably when task vectors exhibit low mutual interference, a condition that naive linear averaging can violate through destructive sign cancellation as more or more divergent task vectors are combined, motivating refinements such as SLERP's norm-preserving spherical interpolation and TIES-merging's trim-elect-sign-and-disjoint-merge procedure for explicitly resolving parameter-level sign conflicts across many simultaneously merged models. Merging's practical appeal, near-zero computational cost relative to any training-based alternative, has driven widespread adoption particularly in open-source ecosystems built around shared pretrained bases and parameter-efficient fine-tuning, but it remains an empirically validated rather than formally guaranteed technique, degrading as more tasks are combined and requiring careful post-merge evaluation across both task-specific and general capability benchmarks before deployment.
Keywords: model merging, task arithmetic, task vectors, TIES-merging, DARE, SLERP, spherical linear interpolation, linear mode connectivity, sign interference, model soups, adapter merging, weight averaging, catastrophic interference, multi-task merging, negation for unlearning
---
## Appendix: Practical Labs
### Lab 1: Task Arithmetic Addition Combines Low-Interference Skills
This lab simulates two fine-tuning runs whose task vectors mostly modify disjoint (low-interference) subsets of parameters, and verifies that adding both task vectors to the shared base recovers strong performance on both tasks simultaneously.
import numpy as np
def test_task_arithmetic_addition_combines_low_interference_skills():
rng = np.random.default_rng(1)
d = 50
block_A = slice(0, 25) # dims relevant to task A
block_B = slice(25, 50) # dims relevant to task B
base = rng.normal(size=d) * 0.1
# each task's fine-tuning mostly moves weights within its own relevant
# block, with only small "bleed" into the other block (imperfect but
# realistic near-orthogonality between unrelated task updates)
finetuned_A = base.copy()
finetuned_A[block_A] += rng.normal(size=25) * 1.0
finetuned_A[block_B] += rng.normal(size=25) * 0.05
finetuned_B = base.copy()
finetuned_B[block_B] += rng.normal(size=25) * 1.0
finetuned_B[block_A] += rng.normal(size=25) * 0.05
tau_A = finetuned_A - base
tau_B = finetuned_B - base
merged = base + tau_A + tau_B
def loss_on_block(weights, target, block):
return np.mean((weights[block] - target[block]) ** 2)
loss_base_A = loss_on_block(base, finetuned_A, block_A)
loss_merged_A = loss_on_block(merged, finetuned_A, block_A)
loss_base_B = loss_on_block(base, finetuned_B, block_B)
loss_merged_B = loss_on_block(merged, finetuned_B, block_B)
print(f"task A loss (relevant block): base={loss_base_A:.4f} merged={loss_merged_A:.4f}")
print(f"task B loss (relevant block): base={loss_base_B:.4f} merged={loss_merged_B:.4f}")
assert loss_merged_A < loss_base_A * 0.1
assert loss_merged_B < loss_base_B * 0.1
print("Task arithmetic addition test passed.")
if __name__ == "__main__":
test_task_arithmetic_addition_combines_low_interference_skills()### Lab 2: Negative Task Vectors Suppress Behavior Without Harming Unrelated Capability
This lab simulates a "toxic" fine-tuned model as base-plus-task-vector along a specific direction, then verifies that subtracting that task vector suppresses the associated behavior below baseline while leaving an orthogonal, unrelated capability direction essentially untouched.
import numpy as np
def test_negative_task_vector_suppresses_behavior():
rng = np.random.default_rng(2)
d = 30
base = rng.normal(size=d) * 0.1
toxic_direction = rng.normal(size=d)
toxic_direction /= np.linalg.norm(toxic_direction)
finetuned_toxic = base + toxic_direction * 1.5
tau_toxic = finetuned_toxic - base
def toxicity_score(weights):
return np.dot(weights, toxic_direction)
score_base = toxicity_score(base)
score_toxic = toxicity_score(finetuned_toxic)
# negating the task vector (with slight over-subtraction) should
# suppress the behavior below even the original base model's level
debiased = finetuned_toxic - tau_toxic * 1.2
score_debiased = toxicity_score(debiased)
print(f"toxicity score: base={score_base:.3f} finetuned_toxic={score_toxic:.3f} "
f"debiased={score_debiased:.3f}")
assert score_toxic > score_base + 1.0
assert score_debiased < score_base
# subtracting the task vector should have little effect on an orthogonal,
# unrelated capability direction
unrelated_direction = rng.normal(size=d)
unrelated_direction -= np.dot(unrelated_direction, toxic_direction) * toxic_direction
unrelated_direction /= np.linalg.norm(unrelated_direction)
def capability_score(weights):
return np.dot(weights, unrelated_direction)
cap_before = capability_score(finetuned_toxic)
cap_after = capability_score(finetuned_toxic - tau_toxic)
print(f"unrelated capability score: before={cap_before:.3f} after={cap_after:.3f}")
assert abs(cap_after - cap_before) < 0.15
print("Task vector negation test passed.")
if __name__ == "__main__":
test_negative_task_vector_suppresses_behavior()### Lab 3: SLERP Preserves Weight Norm Better Than Linear Averaging
This lab compares spherical linear interpolation against naive linear averaging of two same-norm weight vectors at varying angular separation, showing that linear averaging's norm collapses as the angle between models grows, while SLERP preserves the interpolated norm exactly.
import numpy as np
def slerp(v0, v1, t, eps=1e-8):
v0n = v0 / np.linalg.norm(v0)
v1n = v1 / np.linalg.norm(v1)
dot = np.clip(np.dot(v0n, v1n), -1.0, 1.0)
theta = np.arccos(dot)
if theta < eps:
return (1 - t) * v0 + t * v1
sin_theta = np.sin(theta)
norm0 = np.linalg.norm(v0)
norm1 = np.linalg.norm(v1)
interp_norm = (1 - t) * norm0 + t * norm1
direction = ((np.sin((1 - t) * theta) / sin_theta) * v0n
+ (np.sin(t * theta) / sin_theta) * v1n)
return direction * interp_norm
def test_slerp_preserves_norm_better_than_linear_averaging():
rng = np.random.default_rng(3)
d = 40
norm_target = 5.0
results = []
for angle_deg in [10, 60, 120, 170]:
theta = np.deg2rad(angle_deg)
v0 = rng.normal(size=d)
v0 = v0 / np.linalg.norm(v0) * norm_target
rand_vec = rng.normal(size=d)
rand_vec -= np.dot(rand_vec, v0) / np.dot(v0, v0) * v0
rand_vec = rand_vec / np.linalg.norm(rand_vec)
v0n = v0 / np.linalg.norm(v0)
v1n = np.cos(theta) * v0n + np.sin(theta) * rand_vec
v1 = v1n * norm_target
linear_avg = 0.5 * v0 + 0.5 * v1
slerp_mid = slerp(v0, v1, 0.5)
results.append((angle_deg, np.linalg.norm(linear_avg), np.linalg.norm(slerp_mid)))
for angle_deg, lin_norm, slerp_norm in results:
print(f"angle={angle_deg:3d} deg linear_avg_norm={lin_norm:.3f} "
f"slerp_norm={slerp_norm:.3f} target={norm_target}")
assert abs(slerp_norm - norm_target) < 1e-6
lin_norms = [r[1] for r in results]
assert lin_norms == sorted(lin_norms, reverse=True), (
"linear-average norm should shrink as the angle between models grows"
)
assert lin_norms[-1] < norm_target * 0.2
print("SLERP vs. linear averaging norm-preservation test passed.")
if __name__ == "__main__":
test_slerp_preserves_norm_better_than_linear_averaging()### Lab 4: TIES-Merging Resolves Sign Conflicts Better Than Naive Averaging
This lab implements the trim, elect-sign, and disjoint-merge steps of TIES-merging and shows that it recovers a majority-consensus signal far more accurately than naive averaging when a minority task vector has an opposing-sign, comparable-magnitude update.
import numpy as np
def trim(task_vectors, keep_frac=0.5):
trimmed = np.zeros_like(task_vectors)
n_tasks, d = task_vectors.shape
k = max(1, int(d * keep_frac))
for i in range(n_tasks):
idx = np.argsort(-np.abs(task_vectors[i]))[:k]
trimmed[i, idx] = task_vectors[i, idx]
return trimmed
def elect_sign(trimmed):
d = trimmed.shape[1]
elected = np.zeros(d)
for j in range(d):
col = trimmed[:, j]
pos_mass = col[col > 0].sum()
neg_mass = -col[col < 0].sum()
pos_count = np.sum(col > 0)
neg_count = np.sum(col < 0)
if pos_count > neg_count:
elected[j] = 1
elif neg_count > pos_count:
elected[j] = -1
else:
elected[j] = 1 if pos_mass >= neg_mass else -1
return elected
def disjoint_merge(trimmed, elected_sign):
n_tasks, d = trimmed.shape
merged = np.zeros(d)
for j in range(d):
col = trimmed[:, j]
agreeing = col[np.sign(col) == elected_sign[j]]
if len(agreeing) > 0:
merged[j] = agreeing.mean()
return merged
def ties_merge(task_vectors, keep_frac=1.0):
trimmed = trim(task_vectors, keep_frac)
elected = elect_sign(trimmed)
return disjoint_merge(trimmed, elected)
def test_ties_merging_beats_naive_averaging_under_sign_conflict():
rng = np.random.default_rng(4)
n_tasks = 5
d = 50
task_vectors = np.zeros((n_tasks, d))
conflict_dims = range(0, 20)
true_value = 1.0
for j in conflict_dims:
for i in range(n_tasks):
if i < 4:
task_vectors[i, j] = true_value + rng.normal(scale=0.05)
else:
task_vectors[i, j] = -true_value + rng.normal(scale=0.05)
for j in range(20, d):
base_val = rng.normal(scale=0.3)
for i in range(n_tasks):
task_vectors[i, j] = base_val + rng.normal(scale=0.02)
naive_merged = task_vectors.mean(axis=0)
ties_merged = ties_merge(task_vectors, keep_frac=1.0)
naive_err = np.mean(np.abs(naive_merged[list(conflict_dims)] - true_value))
ties_err = np.mean(np.abs(ties_merged[list(conflict_dims)] - true_value))
print(f"mean abs error on conflict dims: naive={naive_err:.4f} TIES={ties_err:.4f}")
assert ties_err < naive_err * 0.3
non_conflict = list(range(20, d))
diff = np.mean(np.abs(naive_merged[non_conflict] - ties_merged[non_conflict]))
print(f"mean abs difference on non-conflict dims: {diff:.4f}")
assert diff < 0.1
print("TIES-merging sign-conflict resolution test passed.")
if __name__ == "__main__":
test_ties_merging_beats_naive_averaging_under_sign_conflict()