red teaming

Red teaming involves adversarial testing to discover model vulnerabilities, weaknesses, and harmful behaviors before deployment. **Purpose**: Find failure modes proactively, test safety guardrails, identify jailbreaks and exploits, stress-test alignment. **Approaches**: **Manual red teaming**: Human experts craft adversarial prompts, explore edge cases, roleplay bad actors. **Automated red teaming**: Models generate attack prompts, search algorithms find vulnerabilities, fuzzing approaches. **Domains tested**: Harmful content generation, bias and fairness, privacy leakage, instruction hijacking, unsafe recommendations. **Process**: Define threat model → generate test cases → attack model → document failures → iterate on mitigations. **Red team composition**: Security researchers, domain experts, diverse perspectives, ethicists. **Findings handling**: Responsible disclosure, prioritize fixes, monitor exploitation. **Industry practice**: Required for major model releases, ongoing process not one-time, bug bounty programs. **Tools**: Garak, Microsoft Counterfit, custom attack frameworks. **Relationship to safety**: Red teaming finds problems, RLHF/constitutional AI address them. Essential for responsible AI development.