validation

A validation set (or holdout set) is a subset of the dataset excluded from training and used to evaluate model performance during training, providing an unbiased estimate of generalization and serving as the key signal for preventing overfitting. Distinction from test set: validation used for hyperparameter tuning and early stopping; test set used ONLY for final evaluation. Overfitting signal: if training loss decreases but validation loss increases, model is memorizing noise (overfitting). Early stopping: stop training when validation metric stops improving for N epochs (patience). Checkpointing: save model weights corresponding to best validation performance, not necessarily expected final epoch. Size: typically 10-20% of data; depends on total dataset size (smaller % for massive data). Stratification: ensure validation distribution matches training/test (e.g., same class balance). Leakage: ensure no data overlap between train and validation (e.g., same user or time period in both). Validation is the compass that guides the training process toward generalizable models.