PPLM (Plug and Play Language Models) is a technique for controllable text generation that steers a pretrained language model's output toward desired attributes (like topic or sentiment) without modifying the model's weights. Instead, it uses small attribute classifiers to guide generation at inference time.

How PPLM Works

• Base Model: Start with a frozen, pretrained language model (like GPT-2).
• Attribute Model: Train a small classifier (often a single linear layer) on the model's hidden states to detect the desired attribute (e.g., positive sentiment, specific topic).
• Gradient-Based Steering: At each generation step, compute the gradient of the attribute model's output with respect to the language model's hidden activations, then shift those activations in the direction that increases the desired attribute.
• Generate: Sample the next token from the modified distribution, which now favors text with the target attribute.

Key Properties

• Plug and Play: The name reflects that you can "plug in" different attribute models without retraining the base LM.
• Composable: Multiple attribute models can be combined — e.g., generate text that is both positive sentiment AND about technology.
• No Weight Modification: The pretrained LM's weights are never changed, preserving its language quality.

Attribute Types

• Sentiment: Steer toward positive or negative tone.
• Topic: Guide generation toward specific subjects (science, politics, sports).
• Toxicity: Steer away from toxic or offensive content.
• Formality: Control the register of generated text.

Limitations

• Slow Generation: Gradient computation at each step significantly slows inference compared to standard sampling.
• Quality Trade-Off: Strong attribute steering can degrade text fluency and coherence.
• Outdated Approach: Modern methods like RLHF, instruction tuning, and prompt engineering achieve better controllability more efficiently.

PPLM was influential in demonstrating that generation could be steered through lightweight, modular classifiers rather than full model retraining.