cross-domain few-shot
**Cross-domain few-shot learning** addresses the challenging scenario where few-shot tasks at test time come from a **different visual or data domain** than the tasks seen during meta-training. It tests whether few-shot learning methods truly learn generalizable learning strategies or merely memorize domain-specific features.
**The Domain Gap Problem**
- **Within-Domain**: Meta-train on ImageNet classes, meta-test on different ImageNet classes. Feature distributions are similar — the model just needs to handle new categories.
- **Cross-Domain**: Meta-train on ImageNet, meta-test on **medical images, satellite imagery, or industrial inspection data**. Feature distributions are fundamentally different — textures, colors, shapes, and visual patterns change entirely.
- **Performance Drop**: Most meta-learning methods see **15–30% accuracy drops** when moving from within-domain to cross-domain evaluation.
**BSCD-FSL Benchmark**
| Target Domain | Dataset | Description | Visual Gap from ImageNet |
|--------------|---------|-------------|--------------------------|
| Agriculture | CropDisease | Plant disease images | Moderate |
| Satellite | EuroSAT | Satellite land use images | Large |
| Medical | ISIC | Skin lesion dermoscopy | Very large |
| Medical | ChestX | Chest X-ray pathology | Very large |
- Performance degrades as the visual gap from the training domain increases.
- ChestX (most different from ImageNet) shows the worst cross-domain performance.
**Why Standard Methods Fail**
- **Domain-Specific Features**: Networks meta-trained on natural images learn features (edges, textures, colors) optimized for that domain. Medical images have entirely different discriminative features.
- **Distribution Shift**: Pixel distributions, spatial frequencies, and channel statistics differ dramatically across domains.
- **Task Structure Mismatch**: The "tasks" in different domains have fundamentally different structures — distinguishing dog breeds vs. distinguishing tissue pathologies.
**Approaches to Cross-Domain Generalization**
- **Large Pre-Trained Backbones**: Models like **CLIP, DINOv2, DeiT** trained on massive diverse datasets learn more universal features that transfer better across domains.
- **Feature-Wise Transformation Layers (FiLM)**: Add learnable scaling and shifting parameters that adapt features to new domains without changing the base network.
- **Domain-Agnostic Representations**: Use adversarial training to learn features that are **domain-invariant** — a domain discriminator cannot tell which domain the features came from.
- **Multi-Source Meta-Training**: Train on episodes from **multiple diverse source domains** simultaneously — increases the diversity of visual experiences.
- **Test-Time Adaptation**: Fine-tune the feature extractor using the support set from the target domain at test time — adapts representations to the new domain on the fly.
- **Self-Supervised Pre-Training**: Methods like contrastive learning capture universal visual structure without domain-specific labels.
**Current Best Practices**
- Start with a **large, diverse pre-trained model** (CLIP, DINOv2).
- Apply **test-time adaptation** using the support set.
- Use **data augmentation** to simulate domain shifts during training.
- Combine metric learning with **support set fine-tuning** for each new task.
Cross-domain few-shot learning is the **true test of meta-learning generalization** — methods that only work within a single visual domain are solving a much easier problem than real-world few-shot learning requires.