plug and play language models (pplm)

**PPLM (Plug and Play Language Models)** is a technique for **controllable text generation** that steers a pretrained language model's output toward desired attributes (like topic or sentiment) **without modifying the model's weights**. Instead, it uses small **attribute classifiers** to guide generation at inference time. **How PPLM Works** - **Base Model**: Start with a frozen, pretrained language model (like GPT-2). - **Attribute Model**: Train a small classifier (often a single linear layer) on the model's hidden states to detect the desired attribute (e.g., positive sentiment, specific topic). - **Gradient-Based Steering**: At each generation step, compute the **gradient** of the attribute model's output with respect to the language model's **hidden activations**, then shift those activations in the direction that increases the desired attribute. - **Generate**: Sample the next token from the modified distribution, which now favors text with the target attribute. **Key Properties** - **Plug and Play**: The name reflects that you can "plug in" different attribute models without retraining the base LM. - **Composable**: Multiple attribute models can be combined — e.g., generate text that is both positive sentiment AND about technology. - **No Weight Modification**: The pretrained LM's weights are never changed, preserving its language quality. **Attribute Types** - **Sentiment**: Steer toward positive or negative tone. - **Topic**: Guide generation toward specific subjects (science, politics, sports). - **Toxicity**: Steer away from toxic or offensive content. - **Formality**: Control the register of generated text. **Limitations** - **Slow Generation**: Gradient computation at each step significantly slows inference compared to standard sampling. - **Quality Trade-Off**: Strong attribute steering can degrade text fluency and coherence. - **Outdated Approach**: Modern methods like **RLHF**, **instruction tuning**, and **prompt engineering** achieve better controllability more efficiently. PPLM was influential in demonstrating that generation could be steered through **lightweight, modular classifiers** rather than full model retraining.