model merging

Model merging combines weights from multiple fine-tuned models to create a single model with combined capabilities. **Methods**: Linear interpolation (weighted average of weights), TIES merging (resolves sign conflicts), DARE (drops and rescales parameters), task arithmetic (add/subtract task vectors). **Use cases**: Combine coding + chat abilities, merge specialized domain models, ensemble without inference overhead. **Process**: Start with models sharing same base architecture, align layers, apply merging algorithm, test extensively as results can be unpredictable. **Popular tools**: mergekit (comprehensive CLI), Hugging Face model merger. **Examples**: WizardLM + CodeLlama merges, Mistral + fine-tunes. **Advantages**: No training required, instant combination, produces single efficient model. **Challenges**: Can cause capability conflicts, quality unpredictable, requires experimentation with merge ratios. **Best practices**: Test component tasks separately, use evaluation suite, try different merge algorithms and ratios, SLERP often works better than linear for very different models. Model merging has become a major technique for creating top open-source models.