critical learning periods

**Critical Learning Periods** in neural networks are **early training phases where the network's future performance and representation quality are largely determined** — exposure to particular data distributions or training conditions during these critical periods has a lasting, often irreversible effect on the final model. **Critical Period Evidence** - **Early Deficit**: Degrading training data quality briefly during early training permanently damages final model performance. - **Late Deficit**: The same degradation later in training has minimal lasting effect — the model recovers. - **Fisher Information**: The Fisher information matrix peaks during critical periods — the network is maximally sensitive to data. - **Representation Crystallization**: Internal representations "crystallize" during critical periods — becoming resistant to change later. **Why It Matters** - **Data Quality**: Ensuring high-quality data during early training is crucial — early corruption causes permanent damage. - **Curriculum Design**: The order and timing of training data exposure matters — not just the data itself. - **Biology Analogy**: Mirrors critical periods in biological development — early sensory experience shapes brain connectivity permanently. **Critical Learning Periods** are **the formative moments of training** — early phases that irreversibly determine the model's representational capacity and final performance.