Continual Learning

What is Continual Learning? Learning new tasks sequentially without forgetting previously learned tasks, enabling models to accumulate knowledge over time.

The Forgetting Problem When training on new tasks, models tend to overwrite weights for old tasks:

Task 1: Learn A, B, C  -->  Model knows A, B, C
Task 2: Learn D, E     -->  Model knows D, E, forgets A, B, C

This is called "catastrophic forgetting."

Approaches to Prevent Forgetting

Regularization Methods Penalize changes to important weights:

# Elastic Weight Consolidation (EWC)
def ewc_loss(model, importance, old_params, lambda_):
    loss = 0
    for name, param in model.named_parameters():
        loss += (importance[name] * (param - old_params[name])**2).sum()
    return lambda_ * loss

# Add to training loss
total_loss = task_loss + ewc_loss(model, fisher, prev_params, 1000)

Replay Methods Store and replay old examples:

class ReplayBuffer:
    def __init__(self, size_per_task=100):
        self.buffer = []
        self.size_per_task = size_per_task

    def add_task(self, task_data):
        samples = random.sample(task_data, self.size_per_task)
        self.buffer.extend(samples)

    def get_replay_batch(self, size):
        return random.sample(self.buffer, size)

Architecture Methods Add new capacity for new tasks:

# Progressive networks: Add new column per task
# PackNet: Prune and freeze for each task
# Modular networks: Route to task-specific experts

Comparison

Metrics

Use Cases

• Chatbots learning from conversations
• Robots adapting to new environments
• Recommendation systems evolving with trends
• Any scenario with sequential data streams

Best Practices

• Evaluate on all tasks, not just current
• Use replay buffers when storage allows
• Consider task similarity for transfer
• Monitor for catastrophic forgetting