Class Weighting
Class weighting is a way of handling skewed classes by multiplying the training loss of each example by a weight that depends on its class, so that mistakes on rare classes count for more and mistakes on common classes count for less. Nothing about the data changes; no rows are copied or deleted. The optimizer simply receives a stronger gradient from the examples it would otherwise ignore, which is a direct attack on the way a plain average loss lets a majority class dominate the objective when the data is imbalanced.
The standard recipe sets each class weight inversely proportional to its frequency, and the arithmetic is simple enough to do by hand. With n examples and k classes, a class holding c examples receives the weight n / (k × c), which is what scikit-learn's class_weight set to balanced computes. For a dataset of 1,000 examples with 990 negatives and 10 positives, the negative weight is 1,000 / (2 × 990), about 0.505, and the positive weight is 1,000 / (2 × 10), exactly 50. Each of the ten positives now pulls with about a hundred times the force of a single negative, so that the two classes contribute equally in total: 990 × 0.505 and 10 × 50 both come to about 500. In PyTorch the same idea appears as the weight argument of CrossEntropyLoss and as pos_weight in BCEWithLogitsLoss, and in Keras as the class_weight argument of fit.
Class weighting is a special case of cost-sensitive learning, in which the weights are meant to encode what each kind of error actually costs. Charles Elkan's 2001 paper "The Foundations of Cost-Sensitive Learning" showed how a cost matrix translates into a decision threshold and into example weights, and the connection explains why inverse-frequency is only a default: it equalizes classes, while the true goal may be that a missed fraud costs two hundred times a false alarm, or the opposite. When real costs are known, they should replace the frequency heuristic, and the weight becomes a business parameter to tune on validation data, often with a grid of values between 1 and the full inverse-frequency ratio.
Inverse frequency overcorrects when classes have very few examples, and a refinement from computer vision addresses this. Cui, Jia, Lin, Song, and Belongie's CVPR 2019 paper "Class-Balanced Loss Based on Effective Number of Samples" argued that additional samples of a class bring diminishing returns because they overlap in what they teach, defining the effective number as (1 − β^n) / (1 − β) for n samples and a hyperparameter β close to 1, and weighting classes by its inverse. In practice this softens the extremes of the long tail, which is why it appears in long-tailed image recognition. The focal loss (Lin and colleagues, 2017) takes a complementary route by weighting individual examples according to how badly the model currently handles them, and it is often combined with class weights.
Compared with resampling, weighting is cheaper and cleaner but not strictly equivalent. It needs no extra memory, creates no synthetic points, and cannot leak duplicates across a train-test split. Yet in stochastic training a weight of 50 on a handful of examples produces a high-variance gradient, since a mini-batch may contain one heavily weighted example or none, and with high weights the optimizer can become unstable or overfit those few rows, in which case a lower learning rate, gradient clipping, or a moderate resampling mix helps. Weights also do not give a model new information about the minority class; they only make it listen harder to what little it has.
The side effect most often overlooked is calibration. Because weighting changes the effective class prior, the model's predicted probability of the rare class is inflated relative to reality, so a raw score of 0.9 no longer means a 90 percent chance. If only ranking or a tuned threshold matters, that is harmless; if the probability itself is used to price risk or trigger an action, it needs recalibration on unweighted validation data. A second misconception is that weights alone solve imbalance: the evaluation still has to use precision, recall, and precision-recall curves rather than accuracy.
Read class weighting as turning the knob on how much each kind of mistake hurts during training: inverse frequency is a sensible default, real error costs are the better target, and the price of the technique is a model whose raw probabilities describe the reweighted world rather than the real one.