prompt tuning
**Prompt Tuning and Prefix Tuning** are the **parameter-efficient fine-tuning methods that prepend small sequences of learnable "soft" token embeddings to the input or intermediate layers** — adapting large pretrained models to downstream tasks without updating any model weights, instead learning a compact set of "virtual tokens" whose embeddings are optimized through backpropagation to steer the frozen model's behavior.
**Prompt Tuning (Lester et al., 2021)**
- Prepend k trainable token embeddings to input sequence.
- Frozen model: All transformer weights stay fixed.
- Only the k × d_model parameters (soft prompt) are trained.
- At inference: Soft prompt tokens + task input → model output.
```python
class SoftPrompt(nn.Module):
def __init__(self, n_tokens=20, d_model=1024):
super().__init__()
# k trainable embeddings (random or vocabulary-initialized)
self.prompt = nn.Parameter(torch.randn(n_tokens, d_model))
def forward(self, input_ids, model):
input_embeds = model.embed_tokens(input_ids) # [B, L, D]
prompt = self.prompt.unsqueeze(0).expand(B, -1, -1) # [B, k, D]
full_input = torch.cat([prompt, input_embeds], dim=1) # [B, k+L, D]
return model(inputs_embeds=full_input)
```
- Key finding: At scale (> 10B params), prompt tuning matches full fine-tuning quality.
- For smaller models (< 1B), performance gap remains vs full fine-tuning.
**Prefix Tuning (Li and Liang, 2021)**
- Extends soft prompts to all transformer layers (not just input).
- Prepend trainable prefix to keys (K) and values (V) at every attention layer.
- Virtual tokens attend to and are attended by all real tokens.
```
For each layer l:
K_l = [P_k^l ; W_k^l · x] # prefix keys prepended
V_l = [P_v^l ; W_v^l · x] # prefix values prepended
Attention uses augmented K_l, V_l → prefix influences all positions
```
- More expressive than input-only prompt tuning → works better for smaller models.
- Trainable parameters: 2 × num_layers × prefix_length × d_model.
- Example: GPT-2 (24 layers, d=1024), prefix=10: ~500K parameters (0.1% of model).
**P-Tuning v1 and v2**
- P-Tuning v1: Insert learnable tokens within input (not just at prefix) + use LSTM to generate soft token embeddings.
- P-Tuning v2: Apply prefix tuning to every transformer layer (similar to prefix tuning) → matches fine-tuning on many NLU benchmarks.
**Comparison of PEFT Methods**
| Method | Where Tokens | Params | Inference Overhead |
|--------|-------------|--------|-----------------|
| Prompt tuning | Input only | k × d | None |
| Prefix tuning | All layers (KV) | 2 × L × k × d | Minor (KV cache) |
| P-Tuning v2 | All layers | Similar to prefix | Minor |
| LoRA | Weight matrices | r × (d_in + d_out) | None (merged) |
| Adapter | After FFN/Attn | 2 × d_adapter × d | Minor |
**Advantages and Limitations**
- **Advantages**: Near-zero inference overhead (soft prompt is tiny), easy task switching (swap prompt), modular.
- **Limitations**: Hard to interpret (soft tokens have no human-readable meaning), less flexible than LoRA for complex adaptations, limited expressiveness at small model scales.
**Applications**
- Multi-task serving: One frozen model + multiple task-specific soft prompts → serve many tasks efficiently.
- Personalization: Per-user soft prompts → personalized assistant behavior without separate models.
- Continual learning: New tasks get new prompts without catastrophic forgetting of model weights.
Prompt tuning and prefix tuning are **the extreme lightweight end of the parameter-efficient fine-tuning spectrum** — by demonstrating that as few as 20 virtual tokens can adapt a frozen trillion-parameter model to new tasks, these methods reveal that pretrained LLMs encode broad latent capabilities that merely need steering, not retraining, offering a glimpse of a future where one set of model weights serves millions of personalized use cases through tiny learned steering vectors rather than millions of separate fine-tuned models.