Parameter-Efficient Fine-Tuning
Parameter-efficient fine-tuning (PEFT) is a family of techniques that adapt a large pretrained model to a new task by training only a small number of added or selected parameters while the original weights stay frozen. Full fine-tuning updates every weight, which for a model with tens of billions of parameters means storing gradients and optimizer state for all of them and saving a complete new copy of the model for each task. PEFT shrinks the trainable portion to a fraction of a percent, so a single set of base weights can serve many tasks, with each task represented by a small file that is loaded on demand.
Low-rank adaptation (LoRA), introduced by Edward Hu and colleagues at Microsoft in 2021, is the most widely used method, and it rests on the idea that the weight change needed for a new task has low intrinsic rank. Instead of updating a large weight matrix W directly, LoRA keeps W frozen and learns two thin matrices, A and B, whose product is added to W's output; with rank r much smaller than the matrix dimensions, the number of trainable values drops dramatically. The paper reported cutting trainable parameters on GPT-3 175B by about 10,000 times and GPU memory by about three times relative to full fine-tuning, with comparable quality. Because the update is a matrix that can be merged into W after training, a LoRA model adds no extra latency at inference, which separates it from methods that insert new layers.
The other main families differ in where the trainable parameters live. Adapter modules, from Houlsby and colleagues in 2019, insert small bottleneck layers inside each transformer block. Prefix tuning (Li and Liang, 2021) prepends learned continuous vectors to the attention keys and values at every layer, while prompt tuning (Lester, Al-Rfou and Constant, 2021) learns embeddings only at the input and showed that this approach becomes competitive with full fine-tuning as model size grows. Further variants include (IA)³, which learns scaling vectors, and bias-only tuning. The Hugging Face PEFT library, released in 2022, put these methods behind a common interface, which helped them spread to everyday practice.
QLoRA, by Tim Dettmers and colleagues in 2023, combined LoRA with quantization to make fine-tuning of very large models feasible on modest hardware. The frozen base model is stored in 4-bit precision using a data type called NormalFloat, with double quantization to compress the scaling constants and paged optimizers to absorb memory spikes, while the LoRA matrices are trained in higher precision. The authors reported fine-tuning a 65-billion-parameter model on a single 48 GB GPU while preserving the performance of 16-bit fine-tuning, and released the Guanaco model family built that way. This put customization of large open models within reach of small labs and individual practitioners.
PEFT is a trade-off, not a free lunch, and its pitfalls are often missed. Small update budgets work well for adjusting style, format and domain vocabulary but can lag full fine-tuning when a task demands large new knowledge or behavior far from pretraining. Hyperparameters such as rank, scaling factor and which layers to target influence results, and the defaults are not universal. Frozen weights also mean less forgetting than full tuning, but not none in the sense of behavior: an adapter can still shift general abilities. It is also a common mistake to think that fewer trainable parameters means a cheaper run: activations still flow through the entire model, so compute per step stays close to the original, and the savings are chiefly in optimizer memory and storage.
Practical value shows up in serving as much as in training. Because each task is a compact delta, one deployment can hold many tenants' or tasks' adapters and switch them per request, and tuned results can be shared as small files rather than full checkpoints. That property, along with low cost, is why PEFT became the default way to customize open-weight models.
Read parameter-efficient fine-tuning as leaving the expensive education untouched and teaching only a thin layer of new habits on top: it keeps one large model and attaches a small, swappable delta for each job.