maml rl
**MAML for RL (Model-Agnostic Meta-Learning for Reinforcement Learning)** applies the MAML meta-learning algorithm to enable RL agents to quickly adapt to new tasks with minimal environment interactions.
## What Is MAML for RL?
- **Goal**: Learn initialization that adapts to new tasks in few gradient steps
- **Method**: Bi-level optimization over distribution of RL tasks
- **Adaptation**: Few episodes (10-100) in new environment
- **Foundation**: Finn et al. 2017 extended to policy gradient methods
## Why MAML for RL Matters
Standard RL requires millions of samples per task. Meta-RL enables robots and agents to adapt to new situations within minutes, not days.
```python
# MAML for RL Algorithm:
for meta_iteration in training:
for task in sampled_tasks:
# Inner loop: adapt to task
policy_adapted = policy.clone()
trajectories = collect_rollouts(policy_adapted, task)
loss = compute_policy_gradient(trajectories)
policy_adapted = policy_adapted - α * grad(loss)
# Outer loop: meta-update
meta_loss = sum(evaluate(policy_adapted, task) for task in tasks)
policy = policy - β * grad(meta_loss, policy)
```
**MAML vs. Other Meta-RL**:
| Method | Adaptation | Memory | Sample Efficiency |
|--------|------------|--------|-------------------|
| MAML | Gradient-based | Low | Good |
| RL² | Recurrent | High | Fast inference |
| PEARL | Latent context | Medium | Very good |