class weight
**Class Weights** is a **technique for handling imbalanced datasets that modifies the loss function to penalize misclassifying the minority class more heavily** — instead of manipulating the data (oversampling or undersampling), class weights make the model "care more" about getting minority examples right by multiplying their loss contribution by a factor inversely proportional to their frequency, so misclassifying 1 fraud case costs as much as misclassifying 100 legitimate ones.
**What Are Class Weights?**
- **Definition**: A modification to the training loss function where each class receives a weight inversely proportional to its frequency — the minority class gets a higher weight (bigger penalty for errors) and the majority class gets a lower weight, making the model optimize equally for both classes despite their unequal representation.
- **The Intuition**: In a dataset with 100 cats and 1 dog, a standard model learns "always predict cat" (99% accuracy). With class weights, misclassifying the dog costs 100× more than misclassifying a cat — forcing the model to actually learn to recognize dogs.
- **No Data Manipulation**: Unlike SMOTE (creates synthetic examples) or undersampling (removes examples), class weights don't change the training data at all — they only change how the loss function weights errors from different classes.
**How Class Weights Work**
| Class | Count | Standard Loss Weight | Balanced Weight | Effect |
|-------|-------|---------------------|----------------|--------|
| Legitimate | 10,000 | 1.0 | 0.05 | Low penalty per error |
| Fraud | 100 | 1.0 | 5.0 | 100× higher penalty per error |
**The Balanced Weight Formula**: $w_c = frac{N}{k imes n_c}$ where N = total samples, k = number of classes, $n_c$ = samples in class c.
For the example above: $w_{fraud} = frac{10100}{2 imes 100} = 50.5$ and $w_{legit} = frac{10100}{2 imes 10000} = 0.505$.
**Implementation Across Frameworks**
| Framework | Code | Notes |
|-----------|------|-------|
| **Scikit-learn** | `LogisticRegression(class_weight='balanced')` | Automatic weight calculation |
| **XGBoost** | `scale_pos_weight=100` | Ratio of negative to positive |
| **PyTorch** | `nn.CrossEntropyLoss(weight=torch.tensor([0.05, 5.0]))` | Manual weight tensor |
| **Keras** | `model.fit(class_weight={0: 0.05, 1: 5.0})` | Dict per class |
| **LightGBM** | `is_unbalance=True` | Automatic handling |
**Class Weights vs Other Imbalance Techniques**
| Technique | Modifies Data? | Modifies Loss? | Pros | Cons |
|-----------|---------------|---------------|------|------|
| **Class Weights** | No | Yes | Simplest, no data change | Can't add new information |
| **SMOTE** | Yes (adds synthetic) | No | Expands decision boundary | Can create noisy examples |
| **Undersampling** | Yes (removes majority) | No | Reduces training time | Loses information |
| **Focal Loss** | No | Yes (down-weights easy examples) | Focuses on hard examples | More complex to tune |
| **Threshold Tuning** | No | No (post-processing) | Adjusts precision/recall after training | Model unchanged |
**The Precision-Recall Trade-off**
| Higher Minority Weight | Effect on Recall | Effect on Precision |
|-----------------------|-----------------|-------------------|
| More aggressive weight | Recall ↑ (catches more minority examples) | Precision ↓ (more false positives) |
| Less aggressive weight | Recall ↓ | Precision ↑ |
| Balanced weight | Good balance | Good balance |
**Class Weights is the simplest and most universally supported technique for handling imbalanced datasets** — requiring just one parameter change (class_weight="balanced") to make any classifier treat minority examples as equally important as majority examples, with the trade-off that it increases recall for the minority class at the cost of some precision, and cannot add new information the way oversampling techniques can.