Curriculum Learning is the training strategy mimicking human education by starting with easier examples and progressively incorporating harder examples — improving convergence speed, generalization, and addressing class imbalance through competence-based sample ordering.

Core Curriculum Learning Concept:

• Educational progression: humans typically learn simple concepts before complex ones; curriculum learning exploits this principle
• Training order matters: presenting examples in appropriate difficulty sequence improves convergence compared to random shuffling
• Competence-based curriculum: difficulty scoring based on model performance metrics enables self-adjusting curricula
• Faster convergence: easier examples provide stable gradient signal early; harder examples refined later
• Better generalization: intermediate difficulty prevents overfitting to easy examples; improves robustness

Difficulty Metrics and Scoring:

• Loss-based difficulty: examples with higher training loss are harder; sort by loss and present in increasing order
• Confidence-based difficulty: examples with lower model confidence are harder; model learns uncertain regions progressively
• Prediction accuracy: examples incorrectly classified are harder; curriculum focuses on challenging regions
• Custom difficulty metrics: task-specific measures (e.g., sentence length for NLP, image complexity for vision)

Self-Paced Learning:

• Learner-driven curriculum: model itself selects which examples to train on based on loss; student chooses curriculum
• Weighting mechanism: dynamically assign sample weights; high-loss examples receive lower weight initially, progressively increase
• Convergence guarantee: theoretically grounded; shows improved generalization under self-paced weighting
• Hyperparameter: learning pace parameter λ controls curriculum progression rate; higher λ transitions faster to harder examples

Curriculum Design Strategies:

• Competence-based: difficulty threshold increases as model improves; achieves higher performance on hard examples
• Time-based: fixed schedule increases difficulty at predetermined milestones regardless of model performance
• Sample-based: curriculum defined over mini-batches; easier samples grouped together for stable early training
• Multi-stage curriculum: pre-define curriculum stages; transition between stages based on validation accuracy plateauing

Hard Example Mining (OHEM):

• Online hard example mining: mine hardest examples from mini-batch; focus optimization on challenging samples
• Hard example ratio: select top-K hard examples (e.g., 25% of batch); balance hard/easy for stable gradients
• Loss ranking: rank by loss; focus on high-loss samples where model makes mistakes
• Benefits: addresses class imbalance; focuses learning on informative examples; improves minority class performance

Applications and Benefits:

• NLP: curriculum learns syntax before semantics; improves performance on downstream language understanding
• Vision: curriculum learns foreground objects before complex scenes; improves robustness to occlusions
• Reinforcement learning: curriculum on task difficulty improves policy learning; enables safe exploration
• Class imbalance: curriculum prioritizes minority class examples; improves underrepresented class performance

Curriculum learning leverages human educational principles — presenting training data in increasing difficulty — to accelerate convergence and improve generalization compared to unordered random shuffling strategies.