cda (counterfactual data augmentation)

**CDA (Counterfactual Data Augmentation)** is a **debiasing technique** that reduces social biases in language models by creating **counterfactual copies** of training data where demographic attributes are swapped. The idea is simple but powerful: if the model sees "The male nurse helped the patient" just as often as "The female nurse helped the patient," it cannot learn a gender association with the nursing profession. **How CDA Works** - **Step 1 — Identify**: Scan training text for mentions of demographic attributes — gendered pronouns (he/she), gendered nouns (king/queen, waiter/waitress), racial terms, names associated with specific demographics, etc. - **Step 2 — Swap**: Create counterfactual copies of each sentence by replacing demographic terms with their counterparts: - "**She** is a talented engineer" → "**He** is a talented engineer" - "**John** received the promotion" → "**Maria** received the promotion" - **Step 3 — Augment**: Add the counterfactual copies to the training set (either replacing originals or supplementing them). - **Step 4 — Train**: Train or fine-tune the model on the augmented dataset. **Types of CDA** - **Gender CDA**: Swap gendered terms (most common and straightforward). - **Name-Based CDA**: Swap names associated with different racial/ethnic groups. - **Multi-Attribute CDA**: Swap terms across multiple bias dimensions simultaneously. **Advantages** - **Intuitive**: The approach is easy to understand and implement. - **Training-Time**: Addresses bias at the source (training data) rather than patching it post-hoc. - **Preserves Task Performance**: Usually maintains or even improves model accuracy since the augmentation provides more diverse training data. **Limitations** - **Incomplete Swaps**: Hard to catch all implicit gender/race signals — names, cultural references, contextual cues may be missed. - **Semantic Validity**: Some swaps create **implausible sentences** (e.g., swapping gendered health conditions). - **Scale**: Doubling the training data increases training cost. - **Binary Limitation**: Simple swap-based CDA treats gender as binary and may not adequately address non-binary identities. CDA is one of the most widely used and accessible debiasing techniques, often combined with other methods like **INLP** or **adversarial debiasing** for comprehensive bias mitigation.