parameter efficient fine tuning peft

**Parameter-Efficient Fine-Tuning (PEFT)** is **the family of techniques that adapts large pre-trained models to downstream tasks by modifying only a small fraction (0.01-5%) of total parameters — achieving comparable performance to full fine-tuning while reducing memory requirements, training time, and storage costs by orders of magnitude**. **LoRA (Low-Rank Adaptation):** - **Mechanism**: freezes the pre-trained weight matrix W (d×d) and adds a low-rank decomposition: ΔW = B·A where A is d×r and B is r×d with rank r ≪ d (typically r=4-64); the forward pass computes (W + ΔW)·x using only r×2×d trainable parameters instead of d² full parameters - **Weight Merging**: at inference, ΔW = B·A is computed once and merged with W, producing zero additional inference latency; the adapted model has identical architecture and speed as the original — no architectural modifications needed at serving time - **Target Modules**: typically applied to attention projection matrices (Q, K, V, O) and optionally MLP layers; applying LoRA to all linear layers (QLoRA-style) with very low rank (r=4) provides broad adaptation with minimal parameters - **QLoRA**: combines LoRA with 4-bit NormalFloat quantization of the frozen base model; enables fine-tuning 65B parameter models on a single 48GB GPU; the base model is quantized (NF4) while LoRA adapters are trained in BF16 **Other PEFT Methods:** - **Adapter Layers**: small bottleneck MLP modules inserted between Transformer layers; each adapter has down-projection (d→r), nonlinearity, and up-projection (r→d); adds ~2% parameters and slight inference latency from additional computation - **Prefix Tuning**: prepends learnable continuous vectors (soft prompts) to the key/value sequences in each attention layer; the model's behavior is steered by these learned prefix embeddings rather than modifying weights; analogous to giving the model a task-specific instruction in its internal representation - **Prompt Tuning**: simpler variant that only prepends learnable tokens to the input embedding layer (not every attention layer); fewer parameters than prefix tuning but less expressive; becomes competitive with full fine-tuning as model size increases beyond 10B parameters - **IA³ (Few-Parameter Fine-Tuning)**: learns three rescaling vectors that element-wise multiply keys, values, and FFN intermediate activations; only 3×d parameters per layer — among the most parameter-efficient methods with competitive performance **Practical Advantages:** - **Multi-Task Serving**: one base model serves multiple tasks by swapping lightweight adapters (2-50 MB each vs 14-140 GB for full model copies); adapter hot-swapping enables serving thousands of personalized models from a single GPU - **Memory Efficiency**: full fine-tuning of Llama-70B requires ~140GB for model + ~420GB for optimizer states + gradients (BF16+FP32); QLoRA reduces this to ~35GB (4-bit model) + ~2GB (LoRA gradients) = single-GPU feasible - **Catastrophic Forgetting**: PEFT methods partially mitigate catastrophic forgetting because the pre-trained weights are frozen; the model retains base capabilities while adapting to the target task through the small adapter parameters - **Training Stability**: fewer trainable parameters produce smoother loss landscapes; PEFT training is typically more stable than full fine-tuning, requiring less hyperparameter tuning and fewer training iterations **Comparison:** - **LoRA vs Full Fine-Tuning**: LoRA achieves 95-100% of full fine-tuning performance for most tasks at r=16-64; gap is larger for tasks requiring significant knowledge update (domain-specific, multilingual); larger rank r closes the gap at the cost of more parameters - **LoRA vs Adapter**: LoRA has zero inference overhead (merged weights); adapters add ~5-10% inference latency from additional forward passes; LoRA is preferred for serving efficiency - **LoRA vs Prompt Tuning**: LoRA is more expressive and consistently outperforms prompt tuning for smaller models (<10B); prompt tuning approaches LoRA performance at very large scale and is simpler to implement PEFT methods, especially LoRA, have **democratized large model fine-tuning — enabling individual researchers and small teams to customize state-of-the-art models on consumer hardware, making the personalization and specialization of billion-parameter models accessible to the entire AI community**.