Continual Learning Catastrophic Forgetting Elastic Weight Consolidation and Task-Incremental Learning
# Continual Learning: Catastrophic Forgetting, Elastic Weight Consolidation, and Task-Incremental Learning
## 1. Introduction & Motivation
Continual learning (also called incremental learning or lifelong learning) addresses the fundamental challenge of learning new tasks sequentially without forgetting previously learned tasks. Unlike standard machine learning where models learn from static datasets, continual learning systems must adapt to new information while maintaining performance on old tasks.
The primary challenge is catastrophic forgetting: when models learn new tasks, their weights change to minimize new task loss, often erasing knowledge of previous tasks. This is especially problematic in:
- Online learning systems requiring real-time adaptation
- Robotics learning new skills without forgetting old capabilities
- Recommendation systems adapting to new user preferences
- Autonomous systems evolving to handle new scenarios
Traditional approaches (e.g., storing all past data) are impractical due to privacy, storage, and computational constraints. Continual learning seeks efficient memory and computation budgets.
This article comprehensively covers continual learning approaches, theoretical analysis, memory-efficient strategies, and practical implementations.
## 2. Core Concepts & Theory
### 2.1 Problem Formulation
Continual learning involves a sequence of tasks
$$ \{T_1, T_2, \ldots, T_K\} $$
, where:
- Task
$$ T_i $$
has training data
$$ (X_i, y_i) $$
available sequentially
- Only current task data is available during learning
- Goal: Minimize overall loss on all tasks
$$L_{ ext{total}} = \sum_{i=1}^{K} L_i( heta)$$
where
$$ heta $$
are model parameters after learning all tasks sequentially.
### 2.2 Catastrophic Forgetting Phenomenon
When learning task
$$ T_j $$
after
$$ T_i $$
, parameters change to minimize
$$ L_j $$
:
$$ heta_j^* = \arg\min_ heta L_j( heta)$$
However, this often increases loss on
$$ T_i $$
:
$$L_i( heta_j^*) > L_i( heta_i^*)$$
Forgetting is quantified as:
$$ ext{Forgetting}_i = L_i( heta_{T}) - L_i( heta_i^*)$$
where
$$ heta_T $$
is final model after learning all T tasks.
### 2.3 Elastic Weight Consolidation (EWC)
EWC prevents forgetting by constraining weights to important parameters:
$$L_j( heta) + \frac{\lambda}{2} \sum_k F_k ( heta_k - heta_k^*)^2$$
where
$$ F_k $$
are importance weights (Fisher information diagonal):
$$F_k = \mathbb{E}_{(x,y) \sim T_i}[( abla_{ heta_k} \log p(y | x))^2]$$
The Fisher information quantifies how much each parameter affects task i predictions. Large
$$ F_k $$
means parameter k is important for task i.
### 2.4 Rehearsal and Experience Replay
Maintain buffer of exemplars from previous tasks:
$$L_j( heta) + \alpha \sum_{i<j} L_i( heta)$$
where exemplar batch from task i is reused. Memory constraint: buffer size
$$ M \ll |D_{ ext{total}}| $$
.
Key question: Which exemplars to store? Strategies:
- Random: Pick uniformly from past tasks
- Uncertainty: Store high-uncertainty samples
- Core-set: Select diverse/representative samples
- Prototypical: Store class prototypes
## 3. Mathematical Formulation
### 3.1 Fisher Information Matrix
The Fisher Information Matrix (FIM) captures parameter importance:
$$F = \mathbb{E}_{(x,y)}[ abla_ heta \log p(y|x) abla_ heta \log p(y|x)^T]$$
Computing full FIM is expensive (
$$ d imes d $$
for d parameters). Diagonal approximation:
$$F_{ ext{diag}} = ext{diag}(\mathbb{E}[ abla_ heta^2 \log p(y|x)])$$
Much cheaper to compute and often sufficient.
### 3.2 EWC Optimization Objective
Combined objective for task j after task i:
$$\mathcal{L} = L_j( heta) + \frac{\lambda}{2} \sum_k F_k^{(i)} ( heta_k - heta_k^{(i)})^2$$
The Lagrangian form shows trade-off between:
- Fitting new task: Minimize
$$ L_j $$
- Preserving old knowledge: Minimize distance to
$$ heta^{(i)} $$
weighted by importance
Learning rate matters: large learning rates may still overwhelm EWC constraint.
### 3.3 Memory Consolidation Theory
Links continual learning to neuroscience. Brain consolidates memories through:
1. Hippocampal encoding (rehearsal)
2. Gradual weight changes
3. Selective parameter importance
EWC mimics this through importance weighting.
### 3.4 Optimal Buffer Management
With memory budget M, which samples to retain?
Coreset selection: Minimize loss on held-out validation set:
$$\mathcal{S}^* = \arg\min_{|\mathcal{S}| \leq M} \sum_{i=1}^{K} \mathcal{L}_i(\mathcal{D}_{ ext{new}} \cup \mathcal{S})$$
Computationally intractable; greedy approximations available:
- K-center: Maximize minimum distance between selected samples
- k-means: Select cluster centers
- Influence functions: Estimate sample importance
## 4. Advanced Theory & Extensions
### 4.1 Synaptic Importance
Generalize beyond Fisher information. Synaptic importance can be computed via:
Gradient-based: Importance ∝ gradient magnitude during training
$$I_k = | abla_{ heta_k} L_i|$$
Hessian-based: Second-order sensitivity
$$I_k = ext{diag}(H_k)$$
where H is Hessian of task loss.
### 4.2 Parameter Isolation
Rather than constraining weights, learn separate parameter subsets per task:
$$ heta = heta^{ ext{shared}} + \sum_{i} m_i \odot heta_i^{ ext{task}}$$
where
$$ m_i $$
are binary masks (trainable or fixed). Benefits:
- No catastrophic forgetting (each task has own parameters)
- Computational efficiency: Reuse shared backbone
- Scaling: Parameter cost grows with tasks
### 4.3 Replay with Generative Models
Generate pseudo-data for past tasks:
$$ heta_j^* = \arg\min_ heta [L_j( heta) + L_i(G( heta))]$$
where G is generative model (VAE, diffusion). Eliminates need to store exemplars.
Advantages:
- Memory efficient (only store generator)
- Scalable to many tasks
Disadvantages:
- Generated data may not reflect true distribution
- Training generator adds computational cost
### 4.4 Online Continual Learning
Data arrives in stream, no batching. Challenge: Memory budget even tighter.
Streaming buffer: Fixed-size FIFO or priority queue:
$$ ext{Memory} = O(M), \quad ext{Update time} = O(1)$$
Trade-off: Smaller buffer → more forgetting, but constant memory.
## 5. Computational Considerations
### 5.1 Fisher Information Computation
Full Hessian:
$$ O(d^2 \cdot |D|) $$
- prohibitive for large models
Diagonal approximation:
- Compute one forward-backward pass per sample
- Cost:
$$ O(d \cdot |D|) $$
- manageable
- Typical: 10-50% overhead vs. standard training
Kronecker-factored: Further approximation reducing cost to O(d)
### 5.2 Memory Overhead
Storing exemplars:
- Buffer size:
$$ M = 1000-10000 $$
typical
- Cost: ~1-100MB depending on data modality
Storing Fisher diagonals:
- Cost: O(d) = 4-400MB for modern models
- Can be compressed (e.g., pruning small values)
Total continual learning overhead: 10-30% typical
### 5.3 Inference Efficiency
Continual learning doesn't increase inference cost:
- No architectural change
- Same forward pass as standard model
- No memory access during inference
### 5.4 Training Time
EWC requires Fisher computation (~20% overhead).
Rehearsal methods:
- Replay buffer: 10-50% additional gradient computation
- Larger batches (mixing new + old data) help amortize cost
## 6. Practical Implementation Strategies
### 6.1 Task Formulation
Disjoint task labels: Each task has own label space
- E.g., Binary classification tasks 1-3 separate models
- Requires knowing task at inference time
Shared label space: All tasks share labels
- More realistic but harder (more negative transfer)
- Requires task-specific heads or meta-learning
Online class-incremental: New classes appear in stream
- No batch boundaries, only new class indicators
- Streaming data with memory constraints
### 6.2 Exemplar Selection Strategies
Herding: Select samples closest to class mean
$$\mathcal{S}_c = \arg\min_s \left\| s - \mu_c ight\|$$
Clustering: Select cluster centers via k-means
$$\mathcal{S}_c = ext{k-means centers}(\mathcal{D}_c, k = |M_c|)$$
Influence-based: Select high-influence samples:
$$I(s) = ext{influence of sample } s ext{ on validation loss}$$
Computed via Hessian-vector products (expensive).
### 6.3 Rehearsal Ratio
When mixing new and old data, what ratio?
$$ ext{Batch} = [1-\alpha] imes ext{new data} + \alpha imes ext{old data exemplars}$$
Typical:
$$ \alpha = 0.3-0.7 $$
(30-70% rehearsal)
Trade-off:
- High
$$ \alpha $$
: Better memory consolidation, slow new learning
- Low
$$ \alpha $$
: Faster new learning, more forgetting
### 6.4 Regularization Strategies
EWC weighting:
$$ \lambda = 0.1-1.0 $$
typical
Large
$$ \lambda $$
: Prevents forgetting but slows new task learning
Small
$$ \lambda $$
: Fast adaptation but more forgetting
Common: Tune
$$ \lambda $$
per task using validation set.
Learning rate scheduling:
$$\eta_j = \eta_0 \cdot 10^{-(j-1)/K}$$
Decay learning rate as tasks increase (new tasks harder to learn).
## 7. Benchmark Datasets & Evaluation
### 7.1 Continual Learning Benchmarks
Permuted MNIST:
- 10 tasks, each: MNIST pixels permuted differently
- 60K training, 10K test per task
- Baseline: ~97% accuracy first task, ~70% last task (no CL)
Split CIFAR-100:
- 10 tasks, each: 10 classes from CIFAR-100
- 50K training, 10K test total
- Baseline: ~80% first task, ~50% last task
CORe50:
- Real-world: Objects in naturalistic scenarios
- 50 object categories, 11 sessions
- Incremental: New objects appear over sessions
Omniglot:
- 1623 character classes, each with 20 samples
- Design: Natural continual learning setting
- Benchmark: Test on sequential tasks
### 7.2 Evaluation Metrics
Backward Transfer (Forgetting):
$$ ext{BWT} = \frac{1}{T-1} \sum_{i=1}^{T-1} (A_{T,i} - A_{i,i})$$
where
$$ A_{i,j} $$
is accuracy on task i after learning task j. Lower is better (less forgetting).
Forward Transfer:
$$ ext{FWT} = \frac{1}{T-1} \sum_{i=2}^{T} (A_{i,i-1} - A_{i,0})$$
Measures how much previous tasks help new tasks.
Average Accuracy:
$$ ext{Avg Acc} = \frac{1}{T} \sum_{i=1}^{T} A_{T,i}$$
Final accuracy on all tasks after sequential learning.
### 7.3 Benchmark Results
Permuted MNIST:
- Naive (no CL): Avg Acc 62%, BWT -27%
- EWC: Avg Acc 72%, BWT -12%
- iCaRL (rehearsal): Avg Acc 75%, BWT -8%
Split CIFAR-100:
- Naive: Avg Acc 42%, BWT -18%
- EWC: Avg Acc 52%, BWT -9%
- Rehearsal (100 exemplars/class): Avg Acc 58%, BWT -5%
- Joint training (oracle): Avg Acc 64%, BWT 0%
## 8. Key Challenges & Limitations
### 8.1 Forgetting vs. Plasticity Trade-off
Core trade-off in continual learning:
- Stability (prevent forgetting): Constrain weights, slow adaptation
- Plasticity (learn new tasks): Allow parameter changes, risk forgetting
No algorithm perfectly balances both. Performance gap to joint training typically 5-15%.
### 8.2 Computational Cost of EWC
Fisher information computation adds 20-50% overhead:
$$ ext{Fisher cost} = O(d \cdot |D|)$$
For large models (1B+ parameters), diagonal Fisher may still be prohibitive.
### 8.3 Memory-Performance Trade-off
Rehearsal methods need memory buffer:
- Buffer size
$$ M = 0 $$
: More forgetting, no memory
- Buffer size
$$ M = 100 $$
: Moderate forgetting, small memory
- Buffer size
$$ M = 1000+ $$
: Little forgetting, significant memory
Optimal buffer size task-dependent, no universal solution.
### 8.4 Task Boundary Issues
Most methods assume clear task boundaries. In reality:
- Task transitions gradual (new and old data mixed)
- No explicit task labels at inference
- Unknown number of future tasks
Streaming continual learning without task boundaries is harder and less studied.
## 9. Hyperparameter Tuning & Optimization
### 9.1 EWC Hyperparameters
Fisher estimation:
- Samples to use: 100-1000 typical
- More samples → more accurate Fisher, higher cost
- Subset sampling common for large datasets
EWC weight:
$$ \lambda \in [0.01, 10] $$
typical
- Specific to dataset and task similarity
- High
$$ \lambda $$
: 10-20% accuracy loss on new task but preserve old
- Low
$$ \lambda $$
: 5% accuracy loss on new task but 30% loss on old
Diagonal approximation:
- Use full Fisher? Too expensive for large models
- Use diagonal only? Standard, works well
- Can also use layer-wise or block-diagonal
### 9.2 Rehearsal Hyperparameters
Buffer size:
$$ M = 100-1000 $$
samples typical
- Scales with task size
- 10-100 exemplars per class standard
- Larger M better accuracy, more memory
Exemplar selection:
- Random: Simple baseline
- Herding: Marginally better, ~1-2% improvement
- Influence-based: 2-3% improvement but expensive
Rehearsal ratio:
$$ \alpha = 0.3-0.7 $$
typical
- High
$$ \alpha $$
(0.7): Preserve old knowledge, slow new learning
- Low
$$ \alpha $$
(0.3): Fast adaptation, more forgetting
### 9.3 Learning Rate Scheduling
Fixed learning rate:
$$ \eta = 1e^{-3} $$
to
$$ 1e^{-4} $$
typical
- Same as standard training
- May need reduction over tasks
Decaying schedule:
$$\eta_j = \eta_0 \cdot 10^{-(j-1)/(K-1)}$$
Reduces learning rate as more tasks arrive.
Adaptive methods: Adam/RMSprop often work better for continual learning than SGD.
### 9.4 Regularization
Batch normalization: Can increase forgetting if not careful
- Use running mean/variance from training
- Freeze batch norm for some continual learning methods
Dropout: Apply to new task learning, not to EWC regularization
- Helps new task generalization
- Typical:
$$ p = 0.2-0.5 $$
## 10. Real-World Applications & Case Studies
### 10.1 Robotics: Learning Sequential Skills
Problem: Robot learns manipulation skills sequentially (grasp, place, push)
Setup:
- Task 1: Grasping (10 training episodes)
- Task 2: Placing (10 training episodes after forgetting grasp)
- Task 3: Pushing (10 training episodes)
- Objective: Maintain all skills
EWC Implementation:
- Fisher computed from final 5 episodes of each task
-
$$ \lambda = 0.4 $$
(balanced between new learning and preservation)
- Buffer: Keep 50 trajectories from previous tasks
Results:
- Success rates: Task 1: 92%, Task 2: 88%, Task 3: 85%
- Without CL: Task 1: 92%, Task 2: 22%, Task 3: 15% (severe forgetting)
- With rehearsal buffer: Task 1: 94%, Task 2: 92%, Task 3: 90%
Key insights:
- Rehearsal critical for robotics (replaying experiences)
- EWC helps preserve basic motor skills
- Combined approach (EWC + rehearsal) best
### 10.2 Online Anomaly Detection
Problem: Detect credit card fraud with evolving patterns
Setup:
- Data stream: 1M daily transactions
- New fraud patterns emerge monthly
- No storing raw transaction data (privacy)
- Memory budget: 10K exemplars
Continual Learning Approach:
- Sliding window rehearsal: Keep recent transactions
- EWC: Constrain weights when retraining
- Monthly retraining with new fraud patterns
Results:
- Detection rate (new fraud): 92% (vs. 88% naive retraining)
- False positive rate: 1.2% (vs. 1.5% naive)
- Processing time: <1ms per transaction (no overhead)
Lessons:
- Privacy-preserving continual learning feasible
- Performance improvements modest but consistent
- Rehearsal buffer crucial for recent patterns
### 10.3 Language Model Adaptation
Problem: Pre-trained language model adapts to domain (medical → legal)
Setup:
- Base model: Trained on general corpus (100B tokens)
- Domain 1: Medical texts (10M tokens)
- Domain 2: Legal texts (10M tokens)
- Constraint: ~1% retraining budget
EWC Fine-tuning:
- Fisher from base model training (low rank approximation)
-
$$ \lambda = 0.5 $$
(preserve base knowledge)
- 1 epoch on domain data
Results:
- Perplexity: Medical domain 35 (vs. 45 naive, 28 oracle)
- Legal domain: 42 (vs. 52 naive, 30 oracle)
- Forward transfer from medical to legal: +3% improvement
Technical details:
- Low-rank Fisher (top 10% parameters)
- Learning rate:
$$ 1e^{-5} $$
(conservative for pre-trained)
- Batch size: 32 (small for efficiency)
### 10.4 Video Object Tracking
Problem: Tracker learns new object classes online
Setup:
- Initial: Trained on common objects (person, car, dog)
- Stream: New object classes appear (rare animals, vehicles)
- Online: Update model within 100ms per frame
- Memory: 1000 frames buffer
Method:
- Replay buffer: Frames with object bounding boxes
- EWC: Diagonal Fisher from previous classes
- Quick adaptation: 1-2 backward passes per new class
Results:
- Tracking accuracy (new class): 85% (vs. 78% naive, 92% offline)
- Tracking latency: 95ms (meets real-time requirement)
- Memory usage: 500MB (within device constraints)
Key techniques:
- Streaming rehearsal (FIFO buffer, replace oldest)
- Fast Fisher approximation (one backward pass)
- Task-aware head: Separate classification head per class
## 11. Integration with Other Methods
### 11.1 Meta-Learning for Continual Learning
Meta-learn learning rates or initialization for fast task adaptation:
$$ heta_j^* = ext{MetaLearner}( heta, D_j, k)$$
Combine with EWC for double benefit:
- Meta-learner provides good initialization
- EWC constraints preserve old knowledge
Performance: 2-5% improvement over EWC alone on Split CIFAR-100.
### 11.2 Contrastive Learning in Continual Setting
Use contrastive objectives (SimCLR) to learn task-invariant features:
$$\mathcal{L}_{ ext{cont}} = -\log \frac{\exp( ext{sim}(z_i, z_j)/ au)}{\sum_k \exp( ext{sim}(z_i, z_k)/ au)}$$
Helps with transfer between tasks, reduces catastrophic forgetting.
### 11.3 Knowledge Distillation
Distill previous task knowledge into new model:
$$\mathcal{L} = \alpha L_j( heta) + (1-\alpha) D_{KL}(p_{ ext{old}}, p_{ ext{new}})$$
where
$$ p_{ ext{old}} $$
is old model output. Helps preserve class boundaries.
### 11.4 Prompt-based Continual Learning
In vision-language models, learn prompts per task:
$$f_{ ext{vision}}( ext{image}) ightarrow ext{CLIP}( ext{prompt}_j)$$
Separate prompts per task, share vision backbone. Eliminates catastrophic forgetting in foundation models.
## 12. Future Research Directions
### 12.1 Theoretical Understanding
Current understanding limited:
- Why does EWC work? Rigorous analysis lacking
- Optimal buffer size? Problem-dependent, no theory
- Trade-off limits? Fundamental barriers unknown
Future directions:
- Convergence analysis for heterogeneous task sequences
- Information-theoretic lower bounds on forgetting
- Optimal memory-performance Pareto frontiers
### 12.2 Efficient Continual Learning at Scale
Current methods limited to small models:
- Vision Transformers: 100M+ parameters, Fisher infeasible
- Large language models: 10B+ parameters, rehearsal memory prohibitive
Research areas:
- Sparse updates (low-rank, adapter modules)
- Task-specific pruning
- Gradient projection to safe subspaces
### 12.3 Real-World Scenarios
Most benchmarks unrealistic:
- Clear task boundaries (real world: gradual transition)
- Uniform task difficulty (real: new hard tasks harder to learn)
- Symmetric negative transfer (real: asymmetric forgetting)
Future: Streaming learning, open-ended task discovery, self-supervised continual learning.
### 12.4 Continual Reinforcement Learning
Adapting agents to changing environments:
- New reward structure changes
- New dynamics (friction, gravity change)
- Sparse reward signals
Combines CL with RL challenges (non-stationarity, credit assignment).
## 13. Summary & Key Takeaways
Core Problem:
- Sequential task learning causes catastrophic forgetting
- Weights change to fit new task, lose old knowledge
- Trade-off between stability (preserve old) and plasticity (learn new)
Elastic Weight Consolidation (EWC):
- Compute Fisher information: Importance of each parameter for old task
- Penalize changes to important parameters:
$$ \lambda F_k ( heta_k - heta_k^*)^2 $$
- Typical:
$$ \lambda = 0.1-1.0 $$
, reduces forgetting by 5-10x
- Overhead: 20% computation for Fisher estimation
Rehearsal Methods:
- Maintain buffer of old task exemplars (~100-1000 samples)
- Mix old exemplars (30-70% of batch) with new task data
- Effective: ~5-8x reduction in forgetting
- Memory trade-off: 10-100MB typical
Exemplar Selection:
- Random: Simple baseline
- Herding/clustering: 1-2% better
- Influence functions: Best but expensive
Hyperparameter Settings:
- EWC weight:
$$ \lambda = 0.1-1.0 $$
(tune per task)
- Fisher samples: 100-1000 (diagonal approximation)
- Buffer size: 10-100 per class (~1000 total typical)
- Rehearsal ratio:
$$ \alpha = 0.3-0.7 $$
(more old data = better stability)
- Learning rate: Usually reduced as tasks increase
Performance Characteristics:
- EWC alone: 70-80% of joint training performance
- Rehearsal alone: 80-90% of joint training
- Combined (EWC + rehearsal): 90-95% of joint training
- Gap-to-oracle usually 5-15%
Current Limitations:
- Requires knowing task boundaries (unrealistic)
- Fisher computation expensive for large models
- Memory buffer needed for rehearsal
- Asymmetric forgetting (hard to model)
Emerging Directions:
- Parameter-isolated methods (task-specific layers)
- Prompt-based approaches for foundation models
- Online continual learning without task boundaries
- Combining with meta-learning for fast adaptation
Continual learning is essential for practical deployed systems, but methods still lag joint training. Recent focus on foundation models (CLIP, GPT) and parameter-efficient fine-tuning shows promise for scalable continual learning.
---
## Appendix: Practical Implementation Labs
### Lab 1: Elastic Weight Consolidation (EWC)
import torch
import torch.nn as nn
import torch.optim as optim
from copy import deepcopy
class EWC:
def __init__(self, model, criterion, device='cpu'):
self.model = model
self.criterion = criterion
self.device = device
self.fisher_matrix = {}
self.optimal_params = {}
def compute_fisher(self, train_loader, num_samples=None):
"""Compute diagonal Fisher Information Matrix"""
self.model.eval()
fisher = {name: torch.zeros_like(param)
for name, param in self.model.named_parameters()}
for batch_idx, (data, target) in enumerate(train_loader):
if num_samples and batch_idx >= num_samples:
break
data, target = data.to(self.device), target.to(self.device)
self.model.zero_grad()
output = self.model(data)
loss = self.criterion(output, target)
loss.backward()
for name, param in self.model.named_parameters():
if param.grad is not None:
fisher[name] += param.grad.data ** 2
# Normalize
for name in fisher:
fisher[name] /= (batch_idx + 1)
return fisher
def register_task(self, train_loader):
"""Register new task: store optimal params and Fisher"""
self.fisher_matrix = self.compute_fisher(train_loader)
self.optimal_params = {
name: param.data.clone()
for name, param in self.model.named_parameters()
}
def train_with_ewc(self, train_loader, epochs=5, lr=0.001, lambda_ewc=0.4):
"""Train on new task while applying EWC regularization"""
optimizer = optim.SGD(self.model.parameters(), lr=lr)
self.model.train()
for epoch in range(epochs):
for data, target in train_loader:
data, target = data.to(self.device), target.to(self.device)
optimizer.zero_grad()
output = self.model(data)
loss = self.criterion(output, target)
# Add EWC term
if self.fisher_matrix:
ewc_loss = 0
for name, param in self.model.named_parameters():
if name in self.fisher_matrix:
fisher = self.fisher_matrix[name]
old_param = self.optimal_params[name]
ewc_loss += (fisher * (param - old_param) ** 2).sum()
loss += lambda_ewc / 2 * ewc_loss
loss.backward()
optimizer.step()### Lab 2: Exemplar Buffer Management
import torch
import numpy as np
class ExemplarBuffer:
def __init__(self, max_size=1000):
self.max_size = max_size
self.exemplars = []
self.labels = []
def add_exemplars(self, data, labels, strategy='herding'):
"""Add exemplars from new task"""
if strategy == 'random':
indices = np.random.choice(len(data),
min(len(data), self.max_size // 10))
elif strategy == 'herding':
indices = self._herding_selection(data,
min(len(data), self.max_size // 10))
else:
indices = range(min(len(data), self.max_size // 10))
for idx in indices:
if len(self.exemplars) < self.max_size:
self.exemplars.append(data[idx].clone())
self.labels.append(labels[idx])
else:
# Replace oldest when buffer full
self.exemplars.pop(0)
self.labels.pop(0)
self.exemplars.append(data[idx].clone())
self.labels.append(labels[idx])
def _herding_selection(self, data, k):
"""Select k samples closest to class mean"""
# Simplified: return indices of k samples
return np.random.choice(len(data), k, replace=False)
def get_batch(self, batch_size=32):
"""Get batch mixing new and exemplar data"""
if len(self.exemplars) > 0:
exemplar_indices = np.random.choice(len(self.exemplars),
min(len(self.exemplars), batch_size))
exemplar_batch = torch.stack(
[self.exemplars[i] for i in exemplar_indices])
exemplar_labels = torch.tensor(
[self.labels[i] for i in exemplar_indices])
return exemplar_batch, exemplar_labels
return None, None### Lab 3: Rehearsal-Based Continual Learning
import torch
import torch.nn as nn
import torch.optim as optim
def train_continual_rehearsal(model, task_loaders, num_epochs=5,
rehearsal_ratio=0.3, device='cpu'):
"""Train on sequence of tasks with rehearsal"""
criterion = nn.CrossEntropyLoss()
buffer = ExemplarBuffer(max_size=1000)
for task_id, train_loader in enumerate(task_loaders):
print(f"Training task {task_id}")
optimizer = optim.Adam(model.parameters(), lr=0.001)
for epoch in range(num_epochs):
for batch_idx, (data, target) in enumerate(train_loader):
data, target = data.to(device), target.to(device)
# Mix with rehearsal buffer
if task_id > 0 and np.random.random() < rehearsal_ratio:
exemplar_data, exemplar_target = buffer.get_batch(len(data))
if exemplar_data is not None:
data = torch.cat([data, exemplar_data])
target = torch.cat([target, exemplar_target])
optimizer.zero_grad()
output = model(data)
loss = criterion(output, target)
loss.backward()
optimizer.step()
# Add exemplars from this task
data_sample = next(iter(train_loader))
buffer.add_exemplars(data_sample[0][:100], data_sample[1][:100])### Lab 4: Forward-Backward Transfer Analysis
def compute_transfer_metrics(model, task_loaders, device='cpu'):
"""Compute forward/backward transfer"""
accuracies = {}
# Evaluate before learning each task
baseline_accs = []
for task_id, _ in enumerate(task_loaders):
acc_matrix = {}
model.eval()
with torch.no_grad():
for eval_task_id, eval_loader in enumerate(task_loaders[:task_id+1]):
correct = 0
total = 0
for data, target in eval_loader:
data, target = data.to(device), target.to(device)
output = model(data)
_, predicted = output.max(1)
correct += (predicted == target).sum().item()
total += target.size(0)
acc = correct / total
acc_matrix[eval_task_id] = acc
accuracies[task_id] = acc_matrix
# Compute metrics
num_tasks = len(task_loaders)
# Backward Transfer (forgetting)
bwt = 0
for i in range(num_tasks - 1):
bwt += accuracies[num_tasks - 1][i] - accuracies[i][i]
bwt /= (num_tasks - 1)
# Forward Transfer
fwt = 0
baseline = 0.5 # Random accuracy
for i in range(1, num_tasks):
fwt += accuracies[i - 1][i] - baseline
fwt /= (num_tasks - 1)
# Average Accuracy
avg_acc = sum(accuracies[num_tasks - 1].values()) / num_tasks
return {'BWT': bwt, 'FWT': fwt, 'Avg_Acc': avg_acc}