Class Weights is a technique for handling imbalanced datasets that modifies the loss function to penalize misclassifying the minority class more heavily — instead of manipulating the data (oversampling or undersampling), class weights make the model "care more" about getting minority examples right by multiplying their loss contribution by a factor inversely proportional to their frequency, so misclassifying 1 fraud case costs as much as misclassifying 100 legitimate ones.
What Are Class Weights?
- Definition: A modification to the training loss function where each class receives a weight inversely proportional to its frequency — the minority class gets a higher weight (bigger penalty for errors) and the majority class gets a lower weight, making the model optimize equally for both classes despite their unequal representation.
- The Intuition: In a dataset with 100 cats and 1 dog, a standard model learns "always predict cat" (99% accuracy). With class weights, misclassifying the dog costs 100× more than misclassifying a cat — forcing the model to actually learn to recognize dogs.
- No Data Manipulation: Unlike SMOTE (creates synthetic examples) or undersampling (removes examples), class weights don't change the training data at all — they only change how the loss function weights errors from different classes.
How Class Weights Work
| Class | Count | Standard Loss Weight | Balanced Weight | Effect |
|---|---|---|---|---|
| Legitimate | 10,000 | 1.0 | 0.05 | Low penalty per error |
| Fraud | 100 | 1.0 | 5.0 | 100× higher penalty per error |
The Balanced Weight Formula: $w_c = frac{N}{k imes n_c}$ where N = total samples, k = number of classes, $n_c$ = samples in class c.
For the example above: $w_{fraud} = frac{10100}{2 imes 100} = 50.5$ and $w_{legit} = frac{10100}{2 imes 10000} = 0.505$.
Implementation Across Frameworks
| Framework | Code | Notes |
|---|---|---|
| Scikit-learn | LogisticRegression(class_weight='balanced') | Automatic weight calculation |
| XGBoost | scale_pos_weight=100 | Ratio of negative to positive |
| PyTorch | nn.CrossEntropyLoss(weight=torch.tensor([0.05, 5.0])) | Manual weight tensor |
| Keras | model.fit(class_weight={0: 0.05, 1: 5.0}) | Dict per class |
| LightGBM | is_unbalance=True | Automatic handling |
Class Weights vs Other Imbalance Techniques
| Technique | Modifies Data? | Modifies Loss? | Pros | Cons |
|---|---|---|---|---|
| Class Weights | No | Yes | Simplest, no data change | Can't add new information |
| SMOTE | Yes (adds synthetic) | No | Expands decision boundary | Can create noisy examples |
| Undersampling | Yes (removes majority) | No | Reduces training time | Loses information |
| Focal Loss | No | Yes (down-weights easy examples) | Focuses on hard examples | More complex to tune |
| Threshold Tuning | No | No (post-processing) | Adjusts precision/recall after training | Model unchanged |
The Precision-Recall Trade-off
| Higher Minority Weight | Effect on Recall | Effect on Precision |
|---|---|---|
| More aggressive weight | Recall ↑ (catches more minority examples) | Precision ↓ (more false positives) |
| Less aggressive weight | Recall ↓ | Precision ↑ |
| Balanced weight | Good balance | Good balance |
Class Weights is the simplest and most universally supported technique for handling imbalanced datasets — requiring just one parameter change (class_weight="balanced") to make any classifier treat minority examples as equally important as majority examples, with the trade-off that it increases recall for the minority class at the cost of some precision, and cannot add new information the way oversampling techniques can.
Related Topics
Explore 500+ Semiconductor & AI Topics
From EUV lithography to CUDA optimization — search the full knowledge base or chat with our AI assistant.