p-tuning

P-Tuning optimizes continuous prompt embeddings for enhanced few-shot and zero-shot performance. **Difference from prompt tuning**: Uses LSTM or MLP to generate prompt embeddings rather than optimizing embeddings directly, provides reparameterization that can improve optimization. **P-Tuning v2**: Adds prompts at each layer of the model, not just input, enables smaller models to match larger model performance, more parameters but still efficient vs full fine-tuning. **Technical approach**: Learnable pseudo-tokens encoded through prompt encoder, resulting embeddings prepended to each transformer layer input (v2), backpropagation trains encoder while freezing base model. **Benefits**: Better optimization landscape than direct embedding tuning, knowledge transfer across tasks, works well for smaller models unlike vanilla prompt tuning. **Use cases**: NLU tasks (classification, NER, QA), few-shot learning, maintaining single model with multiple task adapters. **Comparison**: Prompt tuning (simple, works best for large models), P-tuning (better optimization), P-tuning v2 (deep prompts, best for smaller models), prefix tuning (similar to v2). **Implementation**: Available in PEFT library, relatively straightforward to add to existing architectures.