resolution increase during fine-tuning

**Resolution increase during fine-tuning** is the **practice of pretraining ViT at lower resolution for efficiency and then fine-tuning at higher resolution for accuracy gains** - this two stage workflow improves final performance while keeping total compute manageable. **What Is Resolution Increase Fine-Tuning?** - **Definition**: Train base model at resolution like 224 then continue fine-tuning at higher resolution such as 384. - **Efficiency Rationale**: Most heavy optimization happens at cheaper low resolution. - **Accuracy Rationale**: High resolution fine-tuning adds detail needed for final gains. - **Embedding Issue**: Positional embeddings must be resized to match new token grid. **Why It Matters** - **Cost Control**: Saves significant training compute versus full high resolution training. - **Performance Gain**: Usually improves top-1 accuracy and downstream transfer metrics. - **Flexible Deployment**: Allows one pretrained checkpoint to support multiple inference resolutions. - **Practical Standard**: Widely used in benchmark winning ViT pipelines. - **Transfer Benefit**: Better high detail features for detection and segmentation tasks. **Key Steps in the Workflow** **Stage One Pretraining**: - Train at low resolution with full recipe and regularization. - Learn robust global representation efficiently. **Positional Adaptation**: - Interpolate positional embeddings to new grid shape. - Verify no mismatch in token dimensions. **Stage Two Fine-Tuning**: - Continue training at higher resolution with smaller learning rate. - Use shorter schedule focused on refinement. **How It Works** **Step 1**: Load low resolution checkpoint, resize positional embeddings from old patch grid to new patch grid. **Step 2**: Fine-tune at higher resolution with adjusted learning rate and strong validation monitoring to capture gains without overfitting. **Tools & Platforms** - **timm fine-tune scripts**: Include resize logic for positional embeddings. - **Hugging Face**: Utilities for interpolation and checkpoint adaptation. - **Ablation dashboards**: Compare low to high resolution transfer gains. Resolution increase during fine-tuning is **a high leverage strategy that converts efficient pretraining into high detail final accuracy** - it delivers strong gains with much lower compute than full high resolution from scratch.