Domain-incremental learning is a continual learning scenario where the model's task structure and output space remain the same, but the input data distribution changes across tasks. The model must maintain performance across all encountered domains without forgetting earlier ones.

The Setting

• Task 1: Classify sentiment in product reviews.
• Task 2: Classify sentiment in movie reviews (same output: positive/negative, different input style).
• Task 3: Classify sentiment in social media posts (same output, yet another input distribution).

The output classes don't change, but the characteristics of the input data shift significantly between tasks.

Why Domain-Incremental Learning Matters

• In real deployments, input distributions naturally drift over time — a chatbot encounters different topics, a vision system sees different environments, a medical model encounters patients from new demographics.
• The model must handle any domain it has seen without knowing which domain a test input comes from.

Key Differences from Other Settings

Methods

• Domain-Invariant Representations: Learn features that are robust across domains — domain-adversarial training, invariant risk minimization.
• Replay: Store examples from each domain and replay during training on new domains.
• Normalization Strategies: Use domain-specific batch normalization or adapter layers while sharing the core model.
• Ensemble Methods: Maintain domain-specific expert models with a router that detects the active domain.

Evaluation

• Test on data from all domains after each incremental step.
• No domain/task identifier is provided at test time — the model must perform well regardless of which domain the input comes from.

Domain-incremental learning often benchmarks as easier than class-incremental but more practical — it reflects the realistic scenario of a deployed model encountering gradually shifting data distributions.