peft

**Fine-tuning** is the process of taking a model that has already been pretrained on broad data and training it further on a smaller, targeted dataset so it specializes — adopting a domain's vocabulary, a task's format, or a desired style. **LoRA (Low-Rank Adaptation)** is the most popular *parameter-efficient* way to do it: instead of updating all of a model's billions of weights, you freeze them and train a tiny add-on. The diagram contrasts the two — retraining the whole weight matrix versus learning a small low-rank correction beside it.\n\n```svg ```\n\n**Full fine-tuning updates every weight.** It is the most direct approach and can reach the highest quality, but it is expensive in exactly the way training is: you need optimizer state and gradients for every parameter (several times the model's size in memory), and you end up with a complete, full-size copy of the model for each task you tune. For a large model that means many gigabytes per specialization — costly to train, store, and serve.\n\n**LoRA freezes the model and learns a low-rank patch.** The key observation is that the *change* needed to adapt a model tends to be low-rank — it can be captured by a much smaller matrix. So LoRA leaves the original weight matrix W untouched and learns two skinny matrices, A and B, whose product B·A is added to W at inference: W′ = W + B·A. Only A and B are trained, often well under 1% of the parameters, which slashes memory and produces adapters just megabytes in size.\n\n**QLoRA pushes it onto a single GPU.** QLoRA combines LoRA with a frozen base model quantized to 4-bit, so the bulk of the weights sit in a tiny memory footprint while the small adapters train in higher precision. This is what makes it feasible to fine-tune very large models on modest hardware, and it is a big reason parameter-efficient tuning became ubiquitous.\n\n**Adapters are swappable and composable.** Because a LoRA adapter is small and separate from the base weights, you can keep one frozen base model in memory and hot-swap adapters for different tasks, customers, or styles — even merge an adapter back into the weights for zero inference overhead. Full fine-tuning gives you a monolith per task; LoRA gives you a library of light attachments over a shared backbone.\n\n**Fine-tuning is not the only adaptation tool.** For injecting fresh or proprietary knowledge, retrieval-augmented generation (RAG) or a longer prompt is often better and cheaper, since fine-tuning teaches *behavior and form* more reliably than it memorizes *facts*. The practical decision ladder is usually prompt → RAG → LoRA → full fine-tune, moving down only when the cheaper option is insufficient.\n\n| Approach | Params trained | Artifact per task | Best for |\n|---|---|---|---|\n| Full fine-tuning | ~100% | full checkpoint (GBs) | max quality, big shifts |\n| LoRA | typically <1% | small adapter (MBs) | efficient specialization |\n| QLoRA | <1% + 4-bit base | small adapter | tuning huge models on one GPU |\n| Prompt / RAG | 0% | none / an index | injecting knowledge, fast iteration |\n\nRead fine-tuning through a *what-actually-needs-to-change* lens rather than a *retrain-the-whole-thing* lens: a pretrained model already contains most of the capability, so adaptation is usually a small, low-rank nudge rather than wholesale relearning. LoRA and QLoRA turn that insight into engineering — freeze the expensive part, train a cheap correction — which is why specializing a frontier model went from a data-center job to something that fits on a single GPU and ships as a few-megabyte file.\n