undertraining

**Undertraining** is the **training condition where model has not received enough effective optimization or data exposure to realize its capacity** - it leads to avoidable performance loss despite substantial model size. **What Is Undertraining?** - **Definition**: Model stops before reaching efficient convergence for target tasks. - **Common Causes**: Insufficient token budget, premature stopping, or unstable optimization setup. - **Symptoms**: Large gap between expected and observed performance under fixed architecture. - **Scaling Context**: Frequently seen in parameter-heavy models trained on limited data. **Why Undertraining Matters** - **Capability Loss**: Leaves model performance below achievable frontier for same architecture. - **Cost Inefficiency**: Wastes parameter investment by failing to train capacity adequately. - **Benchmark Weakness**: Can distort comparisons and underestimate architecture potential. - **Roadmap Risk**: Leads to poor strategic conclusions about model family viability. - **Quality**: Undertrained models can show unstable few-shot and long-context behavior. **How It Is Used in Practice** - **Convergence Monitoring**: Track multiple held-out tasks to detect premature stop conditions. - **Token Planning**: Increase effective token budget when loss and capability curves remain steep. - **Optimizer Health**: Stabilize learning-rate and batch schedules to ensure full convergence. Undertraining is **a high-impact source of missed performance potential in model scaling** - undertraining should be diagnosed early because model-size increases cannot compensate for insufficient effective training.