cross-stitch networks

**Cross-stitch networks** is **multi-task networks that learn linear combinations of intermediate task features across branches** - Cross-stitch units dynamically mix representations so tasks share useful signals at learned rates. **What Is Cross-stitch networks?** - **Definition**: Multi-task networks that learn linear combinations of intermediate task features across branches. - **Core Mechanism**: Cross-stitch units dynamically mix representations so tasks share useful signals at learned rates. - **Operational Scope**: It is applied during data scheduling, parameter updates, or architecture design to preserve capability stability across many objectives. - **Failure Modes**: Added mixing parameters increase optimization complexity and may require careful initialization. **Why Cross-stitch networks Matters** - **Retention and Stability**: It helps maintain previously learned behavior while new tasks are introduced. - **Transfer Efficiency**: Strong design can amplify positive transfer and reduce duplicate learning across tasks. - **Compute Use**: Better task orchestration improves return from fixed training budgets. - **Risk Control**: Explicit monitoring reduces silent regressions in legacy capabilities. - **Program Governance**: Structured methods provide auditable rules for updates and rollout decisions. **How It Is Used in Practice** - **Design Choice**: Select the method based on task relatedness, retention requirements, and latency constraints. - **Calibration**: Start with conservative mixing initialization and monitor branch-wise gradient flow during training. - **Validation**: Track per-task gains, retention deltas, and interference metrics at every major checkpoint. Cross-stitch networks is **a core method in continual and multi-task model optimization** - They provide data-driven control over how much sharing occurs at each layer.