gradual unfreezing

**Gradual Unfreezing** is an **alternative name for Progressive Unfreezing** — the fine-tuning strategy where pre-trained layers are incrementally unfrozen from top to bottom over the course of training, preventing catastrophic forgetting while allowing deep adaptation. **Gradual Unfreezing in Practice** - **Identical To**: Progressive Unfreezing. The terms are used interchangeably in the literature. - **Process**: Start with classifier only -> unfreeze one layer group per epoch -> eventually train all layers. - **Key Setting**: The number of epochs per unfreezing phase and the learning rate schedule during each phase. - **Context**: Part of the ULMFiT framework alongside discriminative fine-tuning and STLR. **Why It Matters** - **Robust Transfer**: Prevents the "forgetting cliff" where aggressive fine-tuning destroys useful pre-trained features. - **Curriculum**: Creates a natural curriculum from task-specific (top layers) to general (bottom layers). - **Best Practice**: Recommended for any transfer learning scenario with limited downstream data. **Gradual Unfreezing** is **the same concept as progressive unfreezing** — a careful, layer-by-layer approach to adapting pre-trained models to new tasks.