Gated Fusion is a multimodal fusion mechanism that learns dynamic, input-dependent weights for combining information from different modalities — using sigmoid gating functions inspired by LSTM gates to automatically suppress noisy or uninformative modality channels and amplify reliable ones, enabling robust multimodal inference even when individual modalities degrade.

What Is Gated Fusion?

• Definition: A learned gating network produces scalar or vector weights that control how much each modality contributes to the fused representation, adapting per-sample rather than using fixed combination weights.
• Gate Function: z = σ(W_v·V + W_a·A + b), where σ is the sigmoid function, V and A are modality features, and z ∈ [0,1] controls the mixing ratio.
• Fused Output: h = z ⊙ V + (1−z) ⊙ A, where ⊙ is element-wise multiplication; when z→1 the model relies on vision, when z→0 it relies on audio.
• Adaptive Behavior: Unlike simple concatenation or averaging, gated fusion learns to ignore corrupted modalities — if audio is noisy, the gate automatically reduces its contribution.

Why Gated Fusion Matters

• Robustness: Real-world multimodal data often has missing or degraded modalities (occluded video, background noise); gated fusion gracefully handles these scenarios without manual intervention.
• Efficiency: Gating adds minimal parameters (one linear layer + sigmoid) compared to attention-based fusion, making it suitable for real-time and edge deployment.
• Interpretability: Gate values directly show which modality the model trusts for each input, providing built-in explainability for multimodal decisions.
• Gradient Flow: Sigmoid gates provide smooth gradients during backpropagation, enabling stable end-to-end training of the entire multimodal pipeline.

Gated Fusion Variants

• Scalar Gating: A single scalar z controls the global modality balance — simple but coarse, treating all feature dimensions equally.
• Vector Gating: A vector z ∈ R^d provides per-dimension control, allowing the model to trust different modalities for different feature aspects.
• Multi-Gate Mixture of Experts (MMoE): Multiple gating networks route inputs to specialized expert sub-networks, extending gated fusion to multi-task multimodal learning.
• Hierarchical Gating: Gates at multiple network layers progressively refine the fusion, with early gates handling low-level feature selection and later gates controlling semantic-level combination.

Gated fusion is a lightweight yet powerful multimodal combination strategy — learning input-dependent mixing weights that automatically suppress unreliable modalities and amplify informative ones, providing robust and interpretable multimodal inference with minimal computational overhead.