universal adversarial triggers

**Universal adversarial triggers** are short sequences of tokens that, when prepended or appended to **any input**, reliably cause a language model to produce specific **unwanted behaviors** — such as generating toxic content, making incorrect predictions, or ignoring safety guidelines. Unlike input-specific adversarial examples, these triggers are **input-agnostic** and work across many different prompts. **How They Are Found** - **Gradient-Based Search**: The most common method uses the **HotFlip** or **Autoprompt** algorithm — iteratively replace trigger tokens with candidates that maximize the probability of the target output, using gradient information to guide the search. - **Greedy Coordinate Descent**: Optimize trigger tokens one at a time, testing all vocabulary replacements for each position. - **GCG (Greedy Coordinate Gradient)**: The method used in the influential "Universal and Transferable Adversarial Attacks on Aligned Language Models" paper, combining gradient information with greedy search. **Properties** - **Universality**: A single trigger string works across **many different inputs**, not just one specific example. - **Transferability**: Triggers found on one model often work on **different models**, including black-box APIs. - **Nonsensical Appearance**: Triggers often look like **random gibberish** (e.g., "describing.LaboriniKind ICU proprio") rather than natural language, making them easy to detect but hard to predict. **Examples of Triggered Behavior** - **Jailbreaking**: A trigger suffix causes aligned models to bypass safety training and produce harmful outputs. - **Sentiment Flipping**: A trigger makes a positive review classifier consistently output "negative." - **Targeted Generation**: A trigger causes the model to always generate a specific phrase or topic. **Defenses** - **Perplexity Filtering**: Detect and reject inputs containing high-perplexity (unnatural) token sequences. - **Input Preprocessing**: Paraphrase or tokenize inputs to break trigger patterns. - **Adversarial Training**: Include adversarial examples during safety fine-tuning. - **Ensemble Methods**: Use multiple models and reject outputs when they disagree. Universal adversarial triggers remain one of the most concerning **AI safety vulnerabilities**, demonstrating that aligned language models can be systematically subverted.