Cross-domain few-shot learning addresses the challenging scenario where few-shot tasks at test time come from a different visual or data domain than the tasks seen during meta-training. It tests whether few-shot learning methods truly learn generalizable learning strategies or merely memorize domain-specific features.

The Domain Gap Problem

• Within-Domain: Meta-train on ImageNet classes, meta-test on different ImageNet classes. Feature distributions are similar — the model just needs to handle new categories.
• Cross-Domain: Meta-train on ImageNet, meta-test on medical images, satellite imagery, or industrial inspection data. Feature distributions are fundamentally different — textures, colors, shapes, and visual patterns change entirely.
• Performance Drop: Most meta-learning methods see 15–30% accuracy drops when moving from within-domain to cross-domain evaluation.

BSCD-FSL Benchmark

• Performance degrades as the visual gap from the training domain increases.
• ChestX (most different from ImageNet) shows the worst cross-domain performance.

Why Standard Methods Fail

• Domain-Specific Features: Networks meta-trained on natural images learn features (edges, textures, colors) optimized for that domain. Medical images have entirely different discriminative features.
• Distribution Shift: Pixel distributions, spatial frequencies, and channel statistics differ dramatically across domains.
• Task Structure Mismatch: The "tasks" in different domains have fundamentally different structures — distinguishing dog breeds vs. distinguishing tissue pathologies.

Approaches to Cross-Domain Generalization

• Large Pre-Trained Backbones: Models like CLIP, DINOv2, DeiT trained on massive diverse datasets learn more universal features that transfer better across domains.
• Feature-Wise Transformation Layers (FiLM): Add learnable scaling and shifting parameters that adapt features to new domains without changing the base network.
• Domain-Agnostic Representations: Use adversarial training to learn features that are domain-invariant — a domain discriminator cannot tell which domain the features came from.
• Multi-Source Meta-Training: Train on episodes from multiple diverse source domains simultaneously — increases the diversity of visual experiences.
• Test-Time Adaptation: Fine-tune the feature extractor using the support set from the target domain at test time — adapts representations to the new domain on the fly.
• Self-Supervised Pre-Training: Methods like contrastive learning capture universal visual structure without domain-specific labels.

Current Best Practices

• Start with a large, diverse pre-trained model (CLIP, DINOv2).
• Apply test-time adaptation using the support set.
• Use data augmentation to simulate domain shifts during training.
• Combine metric learning with support set fine-tuning for each new task.

Cross-domain few-shot learning is the true test of meta-learning generalization — methods that only work within a single visual domain are solving a much easier problem than real-world few-shot learning requires.