cross-modal distillation

**Cross-Modal Distillation** is a **knowledge distillation technique that transfers knowledge from one modality to another** — for example, transferring visual knowledge from an image model to a depth-only model, or from a text model to a speech model, enabling inference on a single modality using knowledge from a richer one. **How Does Cross-Modal Distillation Work?** - **Setup**: Teacher trained on modality A (e.g., RGB images). Student trained on modality B (e.g., depth maps). - **Transfer**: Student learns to mimic teacher's representations when both see the same scene from different modalities. - **Paired Data**: Requires paired multi-modal data during training (e.g., RGB + depth pairs). **Why It Matters** - **Sensor Reduction**: Deploy with only a cheap/available sensor (depth camera) while benefiting from knowledge learned on an expensive sensor (RGB camera). - **Multimodal AI**: Enables models that operate on one modality to benefit from another modality's knowledge. - **Applications**: Robotics (RGB teacher -> depth student), medical imaging (MRI teacher -> ultrasound student). **Cross-Modal Distillation** is **knowledge translation between senses** — teaching a model that can only see depth to understand the world as if it could also see color.