multi task learning shared

**Multi-Task Learning (MTL)** is the **training paradigm where a single neural network is trained simultaneously on multiple related tasks (classification, detection, segmentation, depth estimation, etc.) with shared representations — improving generalization by leveraging the inductive bias that related tasks share common features, reducing overfitting on any single task, and enabling efficient deployment where one model replaces many task-specific models at a fraction of the total compute and memory cost**. **Why Multi-Task Learning Works** - **Implicit Data Augmentation**: Each task provides a different view of the same data. Learning to predict depth and surface normals simultaneously forces features to capture 3D structure that benefits both tasks. - **Regularization**: Shared parameters are constrained by multiple loss functions — harder to overfit to any single task's noise. - **Feature Sharing**: Low-level features (edges, textures, shapes) are universal across vision tasks. Sharing these features across tasks avoids redundant computation and enables richer representations. **Architecture Patterns** **Hard Parameter Sharing**: - Shared encoder (backbone), task-specific heads (decoders). - Example: ResNet-50 shared backbone → classification head (FC + softmax), detection head (FPN + RPN + ROI), segmentation head (upsampling + per-pixel classifier). - Advantage: Simple, parameter-efficient, strong regularization. - Risk: Negative transfer — if tasks conflict, shared features compromise both tasks. **Soft Parameter Sharing**: - Each task has its own network, but parameters are regularized to be similar (L2 penalty on weight differences, or cross-stitch networks that learn linear combinations of task features). - More flexible: tasks can learn distinct features where needed while sharing where beneficial. - Cost: More parameters, more memory. **Loss Balancing** The total loss L = Σᵢ wᵢ × Lᵢ requires careful balancing of task weights wᵢ: - **Fixed Weights**: Manually tuned. Fragile — different tasks have different loss scales and convergence rates. - **Uncertainty Weighting (Kendall et al.)**: Learn task weights based on homoscedastic uncertainty. Each weight is 1/(2σᵢ²) where σᵢ is a learned parameter. Tasks with higher uncertainty (harder tasks) receive lower weight — prevents hard tasks from dominating training. - **GradNorm**: Dynamically adjust weights so that all tasks train at similar rates. Monitors gradient norms of each task's loss w.r.t. shared parameters and adjusts weights to equalize them. - **PCGrad (Project Conflicting Gradients)**: When task gradients conflict (negative cosine similarity), project one task's gradient onto the normal plane of the other. Prevents tasks from undoing each other's progress. **Applications** - **Autonomous Driving**: Detect objects + estimate depth + predict lane lines + segment drivable area — all from a shared backbone processing a single camera image. Tesla HydraNet processes 8 cameras with a shared backbone and 48 task-specific heads. - **NLP**: Sentiment analysis + NER + POS tagging + parsing — shared transformer encoder, task-specific classification heads. - **Recommendation**: Click prediction + conversion prediction + dwell time prediction — shared user/item embeddings, task-specific prediction towers. Multi-Task Learning is **the efficiency and generalization paradigm that replaces N separate models with one shared model** — leveraging the insight that real-world tasks share structure, and correctly exploiting that structure produces representations superior to what any single task could learn alone.