model training

An epoch is one complete pass through the entire training dataset, a fundamental unit of training progress. **Definition**: Every example seen exactly once = one epoch. Multiple epochs means multiple passes. **Typical training**: Vision models often train 90-300 epochs. NLP models may train 1-3 epochs (large datasets) or more (small datasets). **LLM pre-training**: Often less than 1 epoch on massive web data. Chinchilla optimal suggests about 1 epoch is ideal. **Multi-epoch considerations**: Later epochs see same data, risk of overfitting. Learning rate schedules often tied to epochs. **Shuffling**: Shuffle data each epoch for better optimization. Different order prevents memorizing sequence. **Steps per epoch**: dataset size / batch size. Common way to measure training progress. **Why multiple epochs**: Limited data requires multiple passes to fully learn patterns. Each pass with different optimization state. **Epoch vs iteration**: Epoch is dataset-level, iteration/step is batch-level. May need thousands of iterations per epoch. **Monitoring**: Track loss per epoch to monitor progress. Compare train vs validation across epochs for overfitting detection.